FINAL REPORT - unicef.org · IDEA International & IDEA-Solutions . iii . of Change. Given resource constraints, hard choices must be made. Although there is no unique route, most

FINAL REPORT

MAPPING EXERCISE ON UNICEF SUPPORT TO

NATIONAL EVALUATION CAPACITY DEVELOPMENT

(NECD)

Submitted to

UNICEF EVALUATION OFFICE

Presented by

IDEA International

(www.idea-international.org)

30 April 2018

http://www.idea-international.org/

http://www.google.ca/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0ahUKEwi0oLCys-vKAhUDqR4KHa8qB_UQjRwIBw&url=http://www.psi.org/partnership/key-development-partners/the-united-nations-childrens-fund-unicef/&psig=AFQjCNG_21beA9pyd1U30loqKPJmmYDoAw&ust=1455132028006531

IDEA International & IDEA-Solutions i

EXECUTIVE SUMMARY

1. National Evaluation Capacity Development (NECD) refers to the process of unleashing, strengthening

and maintaining evaluation capacities, so that development actors progressively engage into significant

and quality evaluation practices and adopt an evaluation culture, thereby contributing to improved

development outcomes. Especially in the context of the 2030 Agenda, evaluations can be used to inform

on progress towards SDGs, to know about current budget allocations and better orient future budget

allocations, to understand better current program performance and improve future policy and

programme design, to hold decision-makers and managers accountable on their use of public monies

and their actions, to demonstrate policy and programme effectiveness, and use them for awareness and

advocacy.

2. This mapping exercise aimed at providing: 1) A clear visual picture of UNICEF-supported NECD

interventions conducted alone or in partnerships along with a characterization of those interventions, 2)

An analysis of the performance of the identified UNICEF-supported NECD interventions based mainly on

a self-assessment; 3) A set of recommendations for the future UNICEF-supported NECD interventions; 4)

A database of the identified UNICEF-supported NECD interventions. This exercise was neither an

evaluation, nor an exhaustive listing of UNICEF-supported NECD intervention. Three data collection

methods were used sequentially to identify and characterize UNICEF-supported NECD interventions: 1)

A documentation review to review key reference documents on NECD; 2) An online survey of UNICEF

M&E staff which led to the identification and characterization of 60 NECD interventions around the world

and a first qualitative assessment of their performance based on their perceptions; and 3) A qualitative

study of eleven NECD interventions selected as case studies, using in-depth interviews with key UNICEF

M&E staff, counterparts, and partners: three at global level: EVALPARTNERS, EVALSDGs, and the Global

Parliamentarians Forum for Evaluation (GPFE); one at regional level: the EAPRO UNDP-UNICEF joint

initiative in the Asia-Pacific Region, and seven at country level: Ecuador-Colombia, Kirghizstan, Morocco,

Nigeria, Philippines, Rwanda, and Sri Lanka.

3. Sixty UNICEF-supported NECD interventions were identified by UNICEF M&E staff in the online survey.

From this list, it appears that the majority of NECD interventions took place at country level (87%), which

is to be expected. Let us note that global interventions (5%) and regional interventions (8%) are far from

negligible, especially if one considers their scope. Country level NECD interventions took place mainly in

South Asia (ROSA – 25%), West and Central Africa (WCARO – 18%), East Asia and the Pacific (EAPRO -

18%), Eastern and Southern Africa (ESARO – 17%), somewhat in Americas and Caribbean (LACRO – 5%),

and little in Europe and Central Asia (ECA – 2%) and Middle East and North Africa (MENA – 2%). Most of

the NECD interventions concerned enabling environment and institutional dimensions of capacity

development. 28,3% were multi-level 1 NECD interventions and 78.3% were multi-dimensional 2

1 Global, regional, country, sector, sub-national levels. 2 Enabling environment, institutional, individual dimensions of capacity development.

IDEA International & IDEA-Solutions ii

interventions, reflecting the importance of upstream work for UNICEF and the tendency to approach

NECD at multiple levels and link them. Sixty percent of the NECD interventions were conducted in middle

income countries. All NECD interventions dealt with the development domain and twenty percent dealt

with both development and humanitarian domains. Government was the main target group of NECD

interventions although in general non-government actors were also involved. By far and large, the 60

UNICEF-supported NECD interventions identified in this mapping exercise have been perceived to have

performed well in terms of relevance, effectiveness, efficiency, and this has been confirmed in the eleven

case studies. However, the performance could be improved in terms of sustainability of NECD seen as a

medium-term process.

4. Good practices encountered include first the capacity to engage a wide variety of international and

national actors, even on sensitive issues, in a respectful and evidence-based way, and progressively build

partnerships. Second, successful NECD interventions benefit a lot from work at all three levels (global,

regional, and country) and the interlinkages between those levels for cross-fertilization. The global level

provides a global perspective and an appreciation of commonalities and differences across regions and

there are trickle down effects at regional and country levels. The regional level provides a benchmarking

across countries in the region, an opportunity to learn from more similar countries, and a conduit of

communication between the global and the country levels. The country level provides concrete evidence

to the regional and global levels on the implementation and results of various NECD interventions in

different country contexts. Country Offices have the local knowledge and adapt to the specific country

conditions, e.g. to the selection of the right entry point and seizing opportunities, the identification of

champions, and the use of local experts. Third, NECD interventions suppose competent interveners. Solid

in-house expertise in evaluation and using quality regional and international evaluation experts make a

world of difference for credibility and results. Fourth, the integration of NECD in strategic plans, work

plans, evaluation frameworks and M&E systems goes a long way in making it a central concern, its

systematization and institutionalization. An NECD intervention should not be seen as an isolated project

per se, but a key building block for promoting and solidifying the evaluation function in the whole

management cycle, and this at all levels. Fifth, UNICEF corporate culture of “putting things on paper”

and documenting experiences in quality publications and briefings helps keep an institutional memory,

disseminate good practices, learn lessons and adjust, and advocate more credibly based on evidence.

5. Lessons learnt include first a recurrent theme that the objectives and scope of any NECD intervention

need to be realistic and take into account a diversity of factors: the slowness of bureaucratic processes

in the public sector, the importance of incentives to vanquish vested interests and resistance to change,

the need to accept the legal, political, institutional and cultural dimensions of evaluation and the

complexity of building successful partnerships. Second, NECD cannot be summed up in a short run

intervention; rather it has to be designed and managed as a medium-term process with a progressive

expansion of the scope and sustainability of results through a programme and a portfolio approach and

an institutionalization perspective. Third, success starts at the phase of design and selection of the NECD

intervention. The objectives, the targets and the roadmap need also to clearly defined with a clear Theory

IDEA International & IDEA-Solutions iii

of Change. Given resource constraints, hard choices must be made. Although there is no unique route,

most often, the first element to address in NECD is the enabling environment, then the institutional and

individual dimensions can be strengthened as clearly identified in the Nigeria case study. Fourth, there

are also some specific lessons depending on the level of capacity-building. At global level, it is often

necessary to “break the silos” and push each partner organization to move beyond its mandate and

rather create a common platform with different perspectives. At institutional level, government

ownership is essential for the establishment of an evaluation culture. At individual level,

professionalization of evaluation is a vital component of NECD.

6. UNICEF has been perceived by the vast majority of the respondents of the online survey and the

qualitative study of case studies as one of the leaders on NECD among international organizations. It

possesses a rich experience in NECD and expertise in NECD at all levels: global, regional, and country.

UNICEF has an uncanny capacity to engage and bring together different types of partners into NECD, i.e.

government, parliamentarians, NGOs, academics, private sector, sub-national actors, international

organizations and other development agencies, etc., to provide a platform for interaction and

information-sharing among all those interested in M&E, and to make linkages across different levels of

NECD. UNICEF is a trusted actor at all levels, recognized for its competencies in evaluation and its role as

an honest broker. Its focus on children is welcome by all, imposes a cross-cutting approach, and is totally

aligned with the 2030 Agenda agreed by all countries. UNICEF is also a relatively strong organization in

evaluation able to intervene from global level to sub-national level through its three-tier organization. It

has a capacity to lead or co-lead interventions at all levels. It is a knowledge-based organization which

can easily transfer skills to the locals with high quality intellectual, ethical and professional standards in

M&E. It can count on a set of competent and committed professionals in evaluation within the

organization and can also mobilize a wide network of professionals, organizations, and networks around

the world who are expert in evaluation and NECD. It has accumulated, documented and disseminated

knowledge on NECD through publications, toolkits, guidelines, etc.

7. In conclusion, UNICEF has accumulated good experience in NECD due to its commitment to SDG and

child focus, working at country, regional and global levels, in partnership with key actors and networks

in evaluation. There is a significant demand of continued leadership from UNICEF by interviewed

counterparts and partners in the NECD intervention case studies. UNICEF brings a unique contribution

to NECD through its child-oriented lens (intersectoral, target group of all social policies key to SDGs) and

an ability to work at global, regional, country and subnational levels and engage a variety of partners in

solid partnerships. Work on NECD also brings visibility and credibility to UNICEF and to its Evaluation

Office. The partial evidence gathered through this mapping exercise would argue for a continued and

institutionalized role for UNICEF to support articulated NECD interventions at global, regional, and

country levels with a child focus in the 2030 Agenda to obtain more sizable results in a more sustainable

way.

IDEA International & IDEA-Solutions iv

8. Six specific recommendations are made (relevant UNICEF actors are indicated in parentheses:

Headquarters – HQ, Regional Offices – ROs, Country Offices - COs):

Recommendation 1 (HQ, ROs): Position NECD in relation with the 2030 Agenda in key strategic documents,

including in the forthcoming UNICEF Evaluation Policy and Regional Evaluation Strategies. This should include

among others a clear definition of NECD and its role for the implementation of the 2030 Agenda, guidance

on approaches and selection criteria to be used by UNICEF approaches; a clear description of the linkages

between global, regional and country level NECD interventions, and the generic roles of UNICEF HQ, RO, CO

as a team and its partners and counterparts. This would contribute to institutionalization and facilitate fund

mobilization.

Recommendation 2 (HQ, ROs): Continue and intensify its partnership approach in NECD at global and

regional levels and take part in promising NECD and related initiatives initiated by other organizations, in

particular with UN organizations, but also bilateral donors and other actors, i.e. initiatives on data generation

and statistics, Voluntary National Reviews, etc. UNICEF has spearheaded several important global and

regional partnerships (EVALPARTNERS, EVALSDGs, GPFE, EAPRO UNDP-UNICEF joint initiative in the Asia-

Pacific Region) which should be continued with added commitments and resources. Such partnerships

provide a more integrated approach, institutional traction, help mobilize resources and go through political

cycles, lead to more sustainable results, and invite reciprocity. MoUs could be signed when relevant with key

partners, especially at global level.

Recommendation 3 (HQ, ROs): Have HQ and ROs support NECD interventions at lower levels3 where those

high-level actors can make a substantial difference, i.e. help deliver concrete milestones and where the

global or regional perspective is eye-opening, stimulates positive competition among countries, enables to

seize economies of scale. A two-tier approach is suggested combining basic services to all country offices

(sharing practices, promoting South-South and triangular cooperation, documenting case-studies with

support from Evaluation Office) and more intense coaching of selected ROs and COs with higher expressed

interest (demand) and high potential for growth in evaluation capacity (strategic).

Recommendation 4 (HQ, ROs, COs): Consider a variety of possible NECD delivery mechanisms, combining

involvement and leadership of internal staff with performance contracts (LTA type) to regional and

international organizations, whether public, private or non-for-profit, that have a track record of supporting

NECD at individual, institutional, and enabling environment levels.

Recommendation 5 (HQ, ROs, COs): Continue improving the capacity of UNICEF to support NECD in a

sustainable way. This may mean more training and coaching of M&E staff, more systematic sharing of good

practices and lessons learnt from more experienced ROs and COs to the still significant number of ROs and

COs that are not so advanced. Some tools for better selection and monitoring and evaluation of NECD

3 Lower levels would mean ROs and COs for HQ and COs for ROs.

IDEA International & IDEA-Solutions v

interventions could also be introduced such as a go/no go grid at country level4 to select NECD interventions

to be supported. Those capacity-building initiatives might be more attractive with financial incentives such

as an innovation fund for new promising NECD interventions to award grants on a competitive basis among

COs.

Recommendation 6 (HQ): Put in place a proper monitoring and documentation system of NECD

interventions on a continued basis. The database of NECD interventions developed within this mapping

exercise can be used as a springboard to develop a web-based information system of the UNICEF-supported

NECD interventions portfolio with decentralized access to a large number of stakeholders and business

intelligence (BI) tools to increase its potential for analysis and its user-friendliness. This data base and

supporting BI tools could be made available to other international organizations to share costs and benefit

from economies of scale as well as obtain a more global perspective on NECD interventions.

44 An example of possible go/no go grid at country level is presented in Annex 6.

IDEA International & IDEA-Solutions vi

ACKNOWLEDGMENTS

The consultants would like to give their sincere thanks to all those that contributed their time and ideas for

this mapping exercise. We would like to thank in particular Mr. George Laryea-Adjei, Director of the UNICEF

Evaluation Office, Ms. Ada Ocampo, Manager of the mapping exercise at the Evaluation Office, all the

members of the Reference Group, in particular Mr. Brook Boyer, Ms. Dorothy Lucks, and Mr. Riccardo

Polastro and, more generally all colleagues from UNICEF and partner organizations that participated in data

collection and discussion of results.

IDEA International & IDEA-Solutions vii

VERBATIM FROM RESPONDENTS

“Proactive involvement of UNICEF has led to more focus on UNICEF concerns.”

“UNICEF is the main agency supporting NECD globally, please continue the good work.”

“We would like to express our deep appreciation to UNICEF who has been the sole development agency that

has been consistently providing support to NECD activities over several years, which was instrumental in the

achievement of most of the outcomes and impact and has shaped the NECD situation in the country. “

IDEA International & IDEA-Solutions viii

TABLE OF CONTENTS

SECTION 1: CONTEXT AND OBJECTIVES ................................................................................................... 1

1.1. CONTEXT ...................................................................................................................................... 1

1.2. OBJECTIVES, USERS, AND SCOPE ................................................................................................... 2

SECTION 2: ANALYTICAL FRAMEWORK ................................................................................................... 4

2.1. DEFINITION OF NATIONAL EVALUATION CAPACITY DEVELOPMENT ................................................ 4

2.2. CHARACTERIZATION AND PERFORMANCE OF NECD INTERVENTIONS ............................................. 5

2.3. NECD STAKEHOLDERS ................................................................................................................... 7

SECTION 3: METHODOLOGY AND IMPLEMENTATION ............................................................................ 10

3.1. OVERALL PROJECT IMPLEMENTATION ......................................................................................... 10

3.2. DATA COLLECTION METHODS ..................................................................................................... 10

3.3. DATA ANALYSIS METHODS.......................................................................................................... 13

3.4 LIMITATIONS ............................................................................................................................... 13

SECTION 4: MAJOR FINDINGS ............................................................................................................... 16

4.1. MAPPING OF UNICEF-SUPPORTED NECD INTERVENTIONS ........................................................... 16

4.2. MAJOR CHARACTERISTICS OF UNICEF-SUPPORTED NECD INTERVENTIONS ................................... 18

4.3. PERCEIVED PERFORMANCE OF UNICEF-SUPPORTED NECD INTERVENTIONS ................................. 23

4.4. GOOD PRACTICES AND LESSONS LEARNT ..................................................................................... 26

4.5. COMPARATIVE ADVANTAGE OF UNICEF ...................................................................................... 27

SECTION 5: RECOMMENDATIONS ......................................................................................................... 30

ANNEX 1 – ORGANIZATION OF THE MAPPING EXERCISE ...................................................................................... 33

ANNEX 2: PROPOSED THEORY OF CHANGE FOR NECD ...................................................................................... 35

ANNEX 3 – DATA COLLECTION INSTRUMENTS .................................................................................................. 39

ANNEX 4 – CASE STUDIES OF NECD INTERVENTIONS AT GLOBAL AND REGIONAL LEVELS ........................................... 44

ANNEX 5 – CASE STUDIES AT COUNTRY LEVEL ................................................................................................. 58

ANNEX 6 – EXAMPLE OF POSSIBLE GO/NO GO GRID AT COUNTRY LEVEL .............................................................. 80

IDEA International & IDEA-Solutions ix

ACRONYMS AND ABBREVIATIONS

CD Capacity Development

CO Country Offices

CP Country Programme

CPAP Country Programme Action Plan

CSO Civil Society Organizations

ECOSOC United Nations Economic and Social Council

GA United Nations General Assembly

GPFE Global Parliamentarians Forum for Evaluation

HQ UNICEF Headquarters

IMEP Integrated Monitoring and Evaluation Plan

IOCE International Organization for Cooperation in Evaluation

LIC Low income country

MIC Middle income country

M&E Monitoring and Evaluation

NECD National Evaluation Capacity Development

NGO Non-Governmental Organizations

RO Regional Offices

RBM Results Based Management

TOC Theory of Change

TOR Terms of Reference

UN United Nations

UNDP United Nations Development Programme

UNDG United Nations Development Group

UNEG United Nations Evaluation Group

UNFPA United Nations Population Fund

UNICEF United Nations Children’s Fund

VOPE Voluntary Organization for Professional Evaluation

UN WOMEN United Nations Entity for Gender Equality and the Empowerment of Women

IDEA International & IDEA-Solutions 0

CONTEXT AND

OBJECTIVES 1


SECTION 1: CONTEXT AND OBJECTIVES

1.1. CONTEXT

One major role of the UN is to develop national capacities (UN Charter page 1). Since the early 2000’s,

UNICEF has put significant effort to support internally and externally evaluation and evaluation capacity

development. Let us mention a few of those normative documents. The 2014-2017 Strategic plan

indicates that priority will be given to undertaking strategic evaluations and enhancing the accountability

system, focusing on decision-making and partnerships. Evaluation will support the achievement of

Strategic Plan objectives by generating findings, conclusions and recommendations to inform

management decisions at all levels and across all results, including in humanitarian action

The UNICEF’s revised Evaluation Policy of 2013 indicates that the evaluation function aims to help UNICEF

to continually improve its performance and results, by supporting organizational learning and

accountability. Since the progressive decentralization of the evaluation function, it is carried out at all

levels of the organization, and it is applicable in all contexts, be it in the humanitarian or development

fields. The policy indicates that Evaluation at UNICEF also aims at contributing to National Evaluation

Capacity Development (NECD), in order to strengthen the capacity of a variety of actors to carry out equity

focused evaluation of national development policies and programmes. The Evaluation Policy not only

provided the first corporate framework for NECD but triggered Executive Board Decisions supporting

NECD initiatives. In its session of June 2016, for example, the Executive Board Requested UNICEF, in

collaboration with other entities of the UN to support National Evaluation Capacity Development, in

accordance with the GA Resolution 69/23.

At a more global level, let us also mention United Nations Evaluation Group (UNEG) norms and standards

for evaluation first established in 2005 and revised in 2016. These norms and standards are meant to serve

as key reference for evaluators and strengthen evaluation practices. In particular, Norm 9 on National

Evaluation Capacities states that “The effective use of evaluation can make valuable contributions to

accountability and learning and thereby justify actions to strengthen national evaluation capacities. In line

with the General Assembly Resolution A/RES/69/237 on building capacity for the evaluation of

development activities at the country level, national evaluation capacities should be supported upon the

request of Member States” (page 12).

As part of the agreement on SDGs and the Agenda 2030, it was mentioned that evaluation should be

included in the framework for the post-2015 development agenda from the outset (UNEG). Follow-Up

and Review (FUR) operating at national, regional and global levels are to promote accountability to

citizens, support effective international cooperation in achieving this Agenda and foster exchanges of best

practices and mutual learning. (par 73/ Agenda).


UNICEF has created key success conditions for NECD during the last twenty years at various levels:

• At global level, UNICEF has played and continues to play a key role in EVALPARTNERS, EVALSDGs,

and the Global Parliamentarians Forum for Evaluation (GPFE). UNICEF created EVALPARTNERS

and seats at the management group. UNICEF co-leads EVALSDGs and provides the secretariat to

the GPFE. UNICEF is also engaged into other partnerships such as EvalYouth and global advocacy

activities aimed at highlighting the importance of developing national evaluation capacities as

well as contributing to create enabling environments that will allow to embed evaluation within

national frameworks and systems to achieve the 2030 agenda.

• At regional level, UNICEF Regional Offices have conducted individually and jointly with partners

regional evaluation capacity development initiatives that benefit NECD as well as directly

supporting selected NECD initiatives led by several UNICEF country offices in their region, trying

to do advocacy and promote the alignment of the national efforts to the 2030 agenda.

• At national level, UNICEF Country Offices are supporting governments, civil society organizations,

parliamentarians and other actors to develop national evaluation policies and frameworks, put

together national evaluation agendas, reinforce capacities to advocate for evaluation,

conceptualize and manage country-led evaluations, in alignment with the endorsement of the

commitment to the 2030 agenda.

1.2. OBJECTIVES, USERS, AND SCOPE

This mapping exercise was aimed at generating information on what UNICEF has done and achieved in

terms of NECD mainly since the elaboration of the revised UNICEF evaluation policy in 2013 to 1)

appreciate the range of NECD interventions that UNICEF led or participated in; 2) provide

recommendations for the next phase of UNICEF strategy on NECD. Specifically, this study aimed at

providing:

1) A clear visual picture of UNICEF-supported NECD interventions conducted alone or in partnerships

along with a characterization of those interventions;

2) An analysis of the performance of the identified UNICEF-supported NECD interventions based

mainly on a self-assessment;

3) A set of recommendations for the future UNICEF-supported NECD interventions;

4) A database of the identified UNICEF-supported NECD interventions.

Let us underline that this exercise was neither an evaluation, nor an exhaustive listing of UNICEF-

supported NECD interventions.

The main users of the information generated through the mapping are indicated in Table 1.1.

Table 1.1: Main Users of the Information Generated through the Mapping

Level Main Users

Global Management and staff of the Evaluation Office, Executive Board, partners

Regional Regional Management Teams, regional Level M&E staff, partners

National Country Management Teams, country Level M&E staff, partners


ANALYTICAL

FRAMEWORK 2


SECTION 2: ANALYTICAL FRAMEWORK

2.1. DEFINITION OF NATIONAL EVALUATION CAPACITY DEVELOPMENT

Capacity development is a long-term, endogenous change process that takes place in the context of

ongoing partner and donor efforts to strengthen related systems of management, governance,

accountability and learning to improve development effectiveness (World Bank 1998)5. Although there is

no unique definition of evaluation capacity (Rist et al. 2010 page 1)6, let us define for the purpose of this

report evaluation capacity development as “…a context-dependent, intentional action system of guided

processes and practices for bringing about and sustaining a state of affairs in which high-quality program

evaluation and its appropriate uses are ordinary and ongoing practices within and/or between one or

more organizations/programs/sites” (Stokdill et al. 2002, page 8) 7 . National Evaluation Capacity

Development (NECD) refers to the process of unleashing, strengthening and maintaining evaluation

capacities, so that development actors progressively engage into significant and quality evaluation

practices and adopt an evaluation culture, thereby contributing to improved development outcomes, in

particular those of the 2030 Agenda. This culture implies a greater demand for evaluation perceived by

users as providing useful information as well as a better capacity to select, manage, and conduct

evaluations, and use evaluation results. Evaluations can be used to know about current budget allocations

and better orient future budget allocations, to understand better current program performance and

improve future policy and programme design, to hold decision-makers and managers accountable on their

use of public monies and their actions, to demonstrate policy and programme effectiveness, and use them

for awareness and advocacy. Even though the thrust of NECD is by definition at country level, there are

significant economies of scale and synergies to be obtained from NECD interventions conducted at global

and regional levels.

NECD concerns of course the national government, but also all other development stakeholders, including

Parliament, sub-national governments, NGOs and CSOs, the private sector, academia, international

organizations and donors. NECD is of course conducted at national level, but also at regional and global

levels with strong linkages and synergies between the three levels. It typically involves four interrelated

and dynamic components: developing an Evaluation Policy, setting up the institutional processes, securing

adequate evaluation capabilities, and engaging with partners.

5 World Bank (1988). Evaluation Capacity Development in Africa: Selected Proceedings from a Seminar in Abidjan, OED proceedings, World Bank, Washington, DC. 6 Rist, R., M.H. Boily, and F. Martin (2010). Influencing Change: Evaluation and Capacity Building. World Bank,

Washington, D.C. 7 Stockdill, S.H., Baizerman, M. and Compton, D.W. 2002. “Toward a Definition of Evaluation Capacity Building

Process: A conversation with the Evaluation Capacity Building Literature.” New Directions for Evaluation, No. 93, Spring 2002, pp. 7-26.


It can be useful to differentiate NECD according to its scope: NECD strategy, NECD domains, and NECD

interventions.

Table 2.1: Scope of NECD

Scope Definition

NECD strategy Global UNICEF strategy8 in terms of NECD, outlining objectives and strategies to achieve those

objectives, with the perspective of contributing to the achievement of SDGs at global level

NECD domains Demand, use, leadership, resources, technical capacity, institutional arrangements, values and

standards

NECD

interventions

UNICEF leadership/ participation in advocacy for NECD, NECD actions conducted by UNICEF alone

or in partnership

Source: IDEA except for NECD domains which are based on Lahey’s typology

NECD interventions may apply at global, regional or national, sector and/or sub-national levels. They may

include one or more dimensions of capacity development: enabling environment, institutional, individual.

Annex 2 presents a proposed Theory of Change for each of those dimensions as contextual information.

NECD is related to several other concepts as evaluation is considered as one pillar of the Results-Based

Management (RBM) approach. Planning is essential since evaluation will to a large extent be made as a

function of objectives, performance indicators and targets defined in the planning system. A

programmatic approach allows for tractable evaluations. Performance-informed budgeting means that

future budget allocations will be influenced by the evaluation of future needs and of past performance.

Monitoring is the companion to evaluation, one feeding into another. Finally, evaluation is worthless

without credible evidence, which highlights the importance of statistical systems, data quality, and

functioning information systems, and use of the evaluation results for decision-making, accountability,

and advocacy.

2.2. CHARACTERIZATION AND PERFORMANCE OF NECD INTERVENTIONS

NECD interventions were characterized using the criteria indicated in Table 2.2.

8 Strategy defined as “a careful plan or method for achieving a particular goal usually over a long period of time” (Merriam-

Webster dictionary, http://www.merriam-webster.com/dictionary/strategy).


Table 2.2: NECD Intervention Characterization Criteria

Characterization criterium Categories

Geographic scope of the NECD

intervention

Global, region or country

UNICEF region in which the NECD

intervention took place

Latin America and the Caribbean, Europe and Central Asia, East Asia and the

Pacific, Eastern and Southern Africa, Middle East and North Africa, South

Asia, West and Central Africa Regional Office

Type of country in which the NECD

interventions took place

Middle income country (MIC), low income country (LIC)9, and fragile10)

Target groups of the NECD

intervention

Level of the NECD intervention Enabling environment, institutional, individual11

Domain of work Humanitarian and/or development12

Objectives

Attention to specific cross-cutting

dimensions of development

Human rights, equity, gender equality, and/or SDGs

Cost

Duration

UNICEF role Alone or partnership, leadership or support

Type of partners

The performance of NECD interventions was assessed again the following criteria:

• Extent to which most important NECD interventions are in line with global reference documents;

• Relevance, effectiveness, efficiency, outputs, outcomes, and sustainability

• Factors supporting good performance

• Challenges to good performance

• Lessons learnt

• Good practices

• UNICEF comparative advantage in terms of NECD.

9 As of 1 July 2016, low-income economies are defined as those with a GNI per capita, calculated using the World

Bank Atlas method, of $1,025 or less in 2015; lower middle-income economies are those with a GNI per capita between $1,026 and $4,035; upper middle-income economies are those with a GNI per capita between $4,036 and $12,475 (http://blogs.worldbank.org/opendata/category/tags/news).

10 A fragile state is a low-income country characterized by weak state capacity and/or weak state legitimacy leaving citizens vulnerable to a range of shocks (https://en.wikipedia.org/wiki/Fragile_state).

11 Including possible multi level interventions since linkages across those levels can lead to more synergies. 12 Some interventions cover both humanitarian and development domains.

https://datahelpdesk.worldbank.org/knowledgebase/articles/378832-what-is-the-world-bank-atlas-method

https://datahelpdesk.worldbank.org/knowledgebase/articles/378832-what-is-the-world-bank-atlas-method


2.3. NECD STAKEHOLDERS

NECD stakeholders are presented in Table 2.3.

Table 2.3: NECD Stakeholders

Level Stakeholder

category

Stakeholders

Global UNICEF

Headquarters

• Executive Board

• Evaluation Office

• Programme directors

UNICEF global

partners

• United Nations Evaluation Group (UNEG)

• Evaluation offices of other international organizations such as UNDP, UNFPA,

World Bank, and selected bilateral donors

• International private firms, NGOs and internationally recognized academic

institutions and consultants

• Global evaluation associations or networks such as EVALPARTNERS,

EVALSDGs, IOCE, etc. that often regroup a variety of the above-mentioned

actors

Regional UNICEF Regional

Office

• Regional Management Team

• Regional Evaluation Advisors

Other regional

partners

• Regional Voluntary Organizations for Professional Evaluation (VOPEs):

Regions now have one or more VOPEs regrouping a variety of actors, both on

the demand side (evaluation clients) and the supply side (providers of

evaluation services). Let us note that the definition of regions is not

homogeneous and does not necessarily correspond to the UN regional

division, so regional VOPEs might overlap and a country might be part of

several regions.

• A variety of regional NGOs and CSOs may also participate in the design and

the delivery of NECD activities as well as regional branches of international

NGOs and CSOs.

• Other international organizations and bilateral donors: In particular, UNDP,

UNFPA, World Bank, and regional development banks can be involved in

regional NECD activities in partnership or in complementarity with those of

UNICEF.


Table 2.3: NECD Stakeholders (cont’d)

Level Stakeholder

category

Stakeholders

National UNICEF Country

Offices

• Country Management Team

• M&E unit

• Programs

Government13

• The national government, including possibly the Presidency, the Office of the

Prime Minister, central agencies like the ministry of Economy and Finance,

and the ministry of planning as well as a number of line ministries and

agencies.

• The sub-national governments.

• The Parliament which may include a verification body such as the Auditor’s

General Office.

• The Judiciary Branch, which may also include a verification body

Other national

partners

• National Voluntary Organizations of Professional Evaluators (VOPEs): Most

countries now have one or more VOPE regrouping a variety of actors, both

on the demand side (evaluation clients) and the supply side (providers of

evaluation services).

• The private sector, be it national firms, international companies or private

foundations, may support technically and financially NECD activities. National

consulting firms and individual consultants as well as universities may be

directly involved in the design and delivery of NECD activities.

• A variety of national Non-Governmental Organization (NGOs) and Civil

Society Organizations (CSOs) may also participate in the design and the

delivery of NECD activities as well as national branches of international NGOs

and CSOs.

• Other international organizations and bilateral donors: In particular, UN

country team (UNDP, UNFPA, etc.), World Bank, and regional development

banks can be involved in national NECD activities, individually without any

consultation or coordination, in complementarity or in partnership with those

of UNICEF.

13 The composition of government varies from one country to the next.


METHODOLOGY AND

EXECUTION 3


SECTION 3: METHODOLOGY AND IMPLEMENTATION

3.1. OVERALL PROJECT IMPLEMENTATION

This project started with an inception mission to UNICEF headquarters in September 2017, meeting the

project manager and a number of key informants. An inception report was prepared, discussed and

finalized in November 2017 based on the comments of the Project Manager and the Reference Group.

Then the online survey took place late November and early December 2017. In-depth interviews for the

case studies lasted from late December 2017 till early February 2018. Analysis took place from mid-

January till end February 2018. The presentation of major findings was made on March 1st 2018 at UNICEF

headquarters to the Evaluation Office, and on March 2nd, 2018 to the Reference Group. The draft final

report was produced in early April 2018 and the final version in late April 2018. Those dates are beyond

the deadlines set in the initial work plan. Several reasons explain those delays, including among others an

underestimation of the level of effort to conduct the study, the time necessary for participants from all

over the world to respond, the overlapping with other assignments, and the New Year break. The support

of the Project Manager to provide the contacts, foster a high response rate, and provide rapid response

to questions and issues along the way has proved very valuable.

3.2. DATA COLLECTION METHODS

The mapping exercise built upon three data collection methods conducted sequentially and providing

different kinds of data. First, the documentation review on key reference documents on NECD was used

for two purposes: as background information to the mapping exercise and to identify if UNICEF-supported

NECD interventions were aligned with those key reference documents. Second, the online survey of

UNICEF M&E staff was used to identify the UNICEF supported NECD interventions, characterize them, and

obtain a first assessment of the performance of the NECD interventions by UNICEF M&E staff. Third,

selected NECD interventions were the object of case studies based on in-depth interviews with key

stakeholders, including UNICEF M&E staff, national counterparts, and/or partners.

Review of relevant documentation

Relevant documentation was obtained from the Project Manager, RO evaluation advisers, and CO M&E

staff, as well as counterparts and partners contacted. A review of relevant websites was also conducted.

Online survey of UNICEF CO M&E staff

An online survey was administered to all UNICEF M&E staff at global, regional, and national levels from a

data base provided by UNICEF. A closed questionnaire (see Annex 3) was sent to 248 staff to document

up to three UNICEF-supported NECD. 103 staff, i.e. a response rate of 42% responded. Altogether 60

UNICEF-supported NECD interventions were identified.


NECD Intervention Case studies

Case studies of NECD Interventions were documented at global, regional, and country levels. At global

and regional levels, the criterium for selection of the NECD intervention was the importance of this

NECD intervention in terms of its strategic meaning for UNICEF strategy, partnerships, and resources

allocated to it. Three NECD interventions were selected at global level: EVALPARTNERS, EVALSDGS, and

the Global Parliamentarians Forum for Evaluation (GPFE), and one at regional level: the review of

national evaluation systems and capacities in Asia Pacific for evaluating progress towards SDGs- a joint

UNDP UNICEF initiative.

At country level, one country was selected in each UN region which had been particularly active in NECD:

Kirghizstan, Morocco, Nigeria, Ecuador14, Philippines, Rwanda, and Sri Lanka. In each selected country,

the NECD intervention or combination of related NECD interventions that was most significant was

selected from the list indicated of NECD interventions indicated in the online survey of UNICEF staff.

Hence a total of 11 case studies were documented: 3 at global level, 1 at regional level, and 7 at country

level as indicated in Figure 3.1.

Figure 3.1: List of Case studies

14 Mexico had been initially selected in the LAC region, but the online survey revealed that NECD interventions in

Mexico were mainly conducted by federal government entities without UNICEF participation, while UNICEF Ecuador had conducted an interesting South-South intervention.


The respondents were first the UNICEF NECD intervention project manager and then the counterparts

that s/he identified as most knowledgeable about this intervention. There is a possible risk of bias of

selection by UNICEF officers in their choice of partners to be interviewed. However, let us remind that this

is not an evaluation, but a mapping exercise and the choice of main counterpart was more or less self-

evident most of the time.

An in-depth interview questionnaire using a quantitative-qualitative (Q2) approach was administered in

two phases (see Annex 3). First, the questionnaire was sent by e-mail to the respondents who could fill

up the Section on the Description of the NECD intervention and her/his evaluation of the performance of

the NECD intervention on an ordinal scale. Second, the consultant team called each respondent for a 30

to 60’ interview to discuss and fill up the qualitative part of the questionnaire, including the justification

of the evaluation made of the performance and other questions related to the perception of the

respondent on the NECD intervention and UNICEF’s role.

Main meta data are presented in Figure 3.2.


Figure 3.2: Main Metadata on the Mapping Exercise

3.3. DATA ANALYSIS METHODS

For the documentation review, each document was read, analyzed and a synthesis was prepared. The

review of literature is presented in Annex 2.

Data on the 60 NECD interventions from the online survey were compiled in a data base. A series of tables

and graphs presenting characteristics of those intervention were prepared and a simple GIS using Google

Maps was prepared to position the interventions and to provide the reader with basic characteristics on

each intervention.

The qualitative information was analyzed and synthesized in the form of case studies presented in

Annexes 4 and 5. Excerpts of case studies were also used in the main text to illustrate some of the main

findings.

3.4 LIMITATIONS

Given its objectives, scope, time and budget, the mapping exercise did not pretend to be an evaluation of

NECD interventions. It aimed principally at identifying UNICEF-supported interventions and doing a first

characterization. The appreciation of performance by UNICEF M&E staff and, in the case of case studies,

key counterparts and partners, was based on their qualitative appreciation. It is entirely possible that the

answers given entailed some bias. Also, triangulation of the results was not really considered in as far as

the three methods were used sequentially and aimed at providing different kinds of data. The only

possible triangulation is for the 11 in-depth case studies conducted interviewing UNICEF and national

counterparts. By far and large, the in-depth interviews confirmed the results from the online survey and

enriched it with more qualitative information. A full-fledged evaluation of a sample of NECD interventions

in the population identified in the mapping exercise might be considered by UNICEF in a second phase for

a more objective and thorough assessment.


The mapping was not designed to be exhaustive. First it is limited to the UNICEF-supported NECD

interventions. Second, it is based on the responses of the current UNICEF M&E staff and some

interventions might have been forgotten. However, by far and large, we estimate it represents a fair

representation of the kinds of NECD interventions supported by UNICEF. Due to the large coverage of

UNICEF around the world, it took some time to coordinate meetings with respondents of the in-depth

interviews. Another factor to consider was that data collection fell around the end of the year holidays

which delayed further responses. The support of the Evaluation Office, and the Manager of the mapping

exercise in particular, was essential to have access to key UNICEF staff and counterparts.

.


MAJOR FINDINGS 4


SECTION 4: MAJOR FINDINGS

4.1. MAPPING OF UNICEF-SUPPORTED NECD INTERVENTIONS

Figure 4.1: Mapping the NECD Interventions

Source: UNICEF-IDEA online survey (November 2017)15.

15 See https://www.google.com/maps/d/viewer?mid=1zvecA_d2jMebHK82tX8pc408_85jfS3E&ll=0%2C0&z=3 for full information.

https://www.google.com/maps/d/viewer?mid=1zvecA_d2jMebHK82tX8pc408_85jfS3E&ll=0%2C0&z=3%20


This map includes all the UNICEF-supported NECD interventions identified in the online survey. Global

interventions are located at UNICEF headquarters in New York City. Regional interventions are located at

the regional UNICEF regional office. Country interventions are positioned on the capital location. Let us

underline that this mapping does not pretend to be exhaustive.

Figure 4.2: Percentage of NECD interventions per level and region16

Source: UNICEF-IDEA online survey (November 2017)

From the list of identified NECD interventions, it appears that the majority of NECD interventions took

place at country level (87%), which is to be expected. Let us note that global interventions (5%) and

regional interventions (8%) are far from negligible, especially if one considers their scope. Country level

NECD interventions took place mainly in South Asia (ROSA – 25%), West and Central Africa (WCARO –

18%), East Asia and the Pacific (EAPRO - 18%), Eastern and Southern Africa (ESARO – 17%), somewhat in

Americas and Caribbean (LACRO – 5%), and little in Europe and Central Asia (ECA – 2%) and Middle East

and North Africa (MENA – 2%). Several factors related to demand and supply may explain this result:17 (i)

some countries benefit from more support and resources from UNICEF because their needs are higher;

(ii) the openness to an evaluation culture varies depending on the government and the institutional

16 The global and regional NECD interventions are categorized separately. The country NECD interventions are

regrouped by region. Sub-national interventions are included in the country category. 17 The prevalence and importance of those factors might be investigated in a second phase following this mapping

exercise.


culture; (iii) there may be demand but not necessarily UNICEF is willing, or in a position to respond to

those demands: and (iv) not necessarily all staff at UNICEF at CO and RO might be clear on the importance

of NECD.

4.2. MAJOR CHARACTERISTICS OF UNICEF-SUPPORTED NECD INTERVENTIONS

This sub-section presents the major characteristics of the identified UNICEF-supported NECD

interventions.

Levels of NECD interventions18

Figure 4.3: Percentage of NECD interventions per level


Eighty seven percent of the NECD interventions were conducted at the national level, which is to be

expected. However, a small number of important NECD interventions were conducted at sector, regional,

and global levels. Interestingly, the number of sub-national interventions was the second largest,

indicating both the needs at this level and the openness of UNICEF to work at this level. Also, worth

underlining, 28,3% were multi-level NECD interventions, pointing out at the capacity of UNICEF to link

several levels.

18 Please note that an NECD intervention may have several elements of a given characteristic. So, the total might

be higher than 100%.


Types of country in which the NECD interventions took place

Figure 4.4: Percentage of NECD interventions per type of country


Sixty percent of the NECD interventions supported were conducted in middle income countries, 28% in

low income countries, 15% as fragile countries.19 This is to be expected as low- income countries have

other priority needs to address, such as just keeping peace and order, paying civil servants, and providing

basic public services.

Work domains in which the NECD interventions took place

Figure 4.5: Percentage of work domain in which the NECD took place


19 As already mentioned, one NECD intervention can be categorized in more than one category.


All NECD interventions dealt with the development domain and 20% dealt with both development and

humanitarian domains. This reflects the orientation of UNICEF towards development programmes, but

also the significant and increasing share of interventions in the humanitarian domain with the growing

number and intensity of man-made and nature-related crises.

Target groups of NECD interventions

Figure 4.6: Percentage of Target groups of NECD intervention


Government was the main target group of NECD interventions. However, CSOs, Parliament, Academia

and NGOs were also largely involved, which reflects the capacity of UNICEF to engage a number of

government and non-government actors and build partnerships.

Dimensions of NECD interventions

Figure 4.7: Percentage of Dimensions of NECD



Most of the NECD interventions covered enabling environment and institutional capacity development

while a little less than half dealt with individual capacity development. 78.3% were multi-dimensional

NECD interventions. This reflects the importance of upstream work for UNICEF and the tendency to

approach NECD with multiple dimensions.

Objectives of NECD Interventions

Figure 4.8: NECD interventions by objective


NECD interventions were quite varied and had multiple objectives. The four major ones were to:

contribute to the development of NEPs, build up partnerships to conduct NECD, provide training and

professional development opportunities as well as awareness and advocacy activities for NECD.

Duration of the NECD interventions

Figure 4.9: NECD interventions by duration



The duration of NECD interventions varied a lot. A significant share had a short duration: 27% last 3

months or less, 39% between 3 and 12 months, and 34% more than one year. UNICEF has been able to

seize short run opportunities, but also build up capacities over the medium run.

Alignment with Reference Documents

Figure 4.10: Extent to which the NECD intervention is in line with global reference documents


Most NECD interventions are by far and large aligned with key reference documents. This reflects a good

dissemination of those documents by the UN and UNICEF in particular.

Incorporation of cross-cutting dimensions in NECD interventions

Figure 4.11: Percentage of NECD interventions considering various cross-cutting dimensions



Most NECD interventions took into account cross-cutting dimensions of human rights, equity, gender

equality, and SDGs. The dimensions less accounted for in some cases were human rights and gender

equality. This might reflect less relevance and/or less interest on the part of the national counterpart

government.

4.3. PERCEIVED PERFORMANCE OF UNICEF-SUPPORTED NECD INTERVENTIONS

This sub-section presents a first assessment by UNICEF M&E staff of the 60 identified UNICEF-supported

NECD interventions based on their qualitative perceptions as collected in the online survey. This is

complemented, when deemed relevant, with the results of the case studies, also based on the qualitative

perceptions of UNICEF staff and selected counterparts and partners as obtained from in-depth interviews.

Figure 5.1: Perceived performance of UNICEF-supported NECD interventions


The overall performance of the 60 identified UNICEF-supported NECD interventions is perceived as high

according to the respondents of the online survey. On an ordinal scale of 0 to 3, the grades are from the

highest to the lowest for relevance (2.7), effectiveness (2.3), efficiency (2.3), and sustainability (2.0).20

This result of the online survey is triangulated by the results of the case studies (see Annexes 4 and 5).

The relevance of UNICEF supported NECD interventions has been systematically rated as high. The global

interventions are directly supportive of NECD. EVALPARTNERS aims at strengthening national evaluation

capacities through partnerships and advocacy activities. EVALSDGs promotes evaluation activities around

the SDGs and orients and supports the review and follow-up process on the SDG 2030 agenda by adding

value through evaluation. The Global Parliamentarians Forum for Evaluation (GPFE) aims at strengthening

the demand and the use of evaluation by parliamentarians, creating enabling environments for nationally

20 The measurement of final outcomes and impacts fell outside of the scope of this mapping exercise, mainly because

many interventions were too recent.


owned, transparent, systematic evaluation processes, and promote national evaluation policies and

systems. The regional interventions also support national capacities directly, but also indirectly by

fostering experience exchanges, dissemination of good practices and lessons learnt, training on evaluation

approaches and methodologies, and advocacy for important evaluation issues. For example, the EAPRO

UNDP-UNICEF joint initiative and country case studies in the Asia-Pacific region21 aim at generating

knowledge to guide NECD for the 2030 Agenda, identifying successes and lessons in terms of national

evaluation systems, fostering peer learning and providing global and regional NECD guidance through the

production of readiness assessments at country level. At country level, NECD interventions have

supported the establishment and strengthening of national Monitoring & Evaluation systems (cases of

Kyrgyzstan and Sri Lanka), pushed for important elements of NECD such as the formulation of a National

Evaluation Policy (NEP) (case of Philippines) or of an Evaluation Plan (case of Ecuador with support from

Colombia), and of course building up the individual competencies and skills of a variety of actors, public

sector officials and professionals, parliamentarians, NGO and CSO managers and professionals,

academics, and consultants. Many NECD interventions supported the creation and strengthening of a

VOPE (cases of Morocco, Rwanda) to strengthen the culture of evaluation, increase national capacities for

policy and programme evaluation, and conduct advocacy for the institutionalization of the evaluation

function.

The effectiveness and to a large extent the efficiency in producing outputs have also been perceived as

high.22 The global and regional NECD interventions resulted in the establishment and support of several

global networks and partnerships, the preparation of various materials and platforms for communication,

learning and advocacy, a number of evaluations, publications and documentation of NECD interventions.

EVALPARTNERS has clearly strengthened the enabling environment for evaluation by increasing the

demand and the use of evaluation and demonstrated the power of partnerships for influencing global

agendas. EVALPARTNERS has proven to be a very good NECD intervention to foster global NECD initiatives

and partnerships for a more coordinated NECD. It has offered a positive ecosystem and institutional

framework for several networks and NECD interventions. EVALSDGs has also produced with little money

a wide variety of outputs helpful for the preparation towards the evaluation of the SDGs, including

important publications, the spotlight initiative, organization of regional consultations, advocacy for the

establishment of frameworks for SDGs evaluation and how to influence the Voluntary National Reviews

(VNRs). Through capacity building, knowledge sharing and advocacy, the GPFE has empowered

parliamentarians to deliver evidence-based policies and thus contributed to good governance, sustainable

development and social equity. At regional level, the recent EAPRO UNDP-UNICEF joint initiative in the

Asia-Pacific Region produced two country diagnostics, but also engaged the participating countries into a

stimulating debate about the role of M&E systems, and evaluations in particular, to monitor, review and

evaluate implementation of 2030 Agenda and recommendations to strengthen national evaluation

capacity for the SDGs with a demonstrating and stimulating effect on other countries of the region.

21 Called EAPRO UNDP-UNICEF initiative in the Asia-Pacific Region in the rest of the report. 22 The details of outputs and immediate outcomes for each global or regional case study are presented in Annex 4.


At country level, the outputs of the NECD varied from one country to another23. However, the most often

encountered outputs in the seven case studies (Ecuador-Colombia, Kyrgyzstan, Morocco, Nigeria,

Philippines, Rwanda, and Sri Lanka) were the creation and strengthening of evaluation institutions such

as a VOPE and a joint evaluation committee, the elaboration and adoption of a National Evaluation Policy

(NEP), national evaluation framework and evaluation plans, the improved evaluation knowledge and skills

of various key actors as a result of training and exchange of experiences, the production of learning and

advocacy material and platforms, evaluation reports, publications, and documentation of NECD

interventions. Beyond the important production of outputs, the participatory process of engaging a

variety of actors at national, sub-national, regional, and international levels is worth mentioning.

Those outputs have translated already into immediate outcomes in a number of cases. The main

outcomes at global and regional levels were the following: putting evaluation on the agenda of work of

many actors, introducing evaluation in global and regional policy-making frameworks, promoting a culture

of evaluation in global and regional policies and programs, and building up global and regional resources,

momentums and synergies for evaluation. At country level again, the outcomes varied from one country

to the next. However, the most often encountered outcomes were the introduction of evaluation in

policy-making frameworks, the promotion of a culture of evaluation in national policies and programs,

the progressive development of a stronger evaluation function and M&E systems, the capacity building of

national resources and further development of partnerships.

Sustainability is probably the lowest rated criteria in the online survey while still rated as rather good (2

on a scale of 3). This concern for sustainability was also noted in most case studies. A number of NECD

interventions are short run, with no guarantee of continued engagement of UNICEF and its partners as

well as availability of resources. Ad hoc interventions can be a way to seize an opportunity, but also to

scatter resources into too many activities, not to mention they complicate follow up and reporting. It is

hard to keep up the momentum both for UNICEF staff and for counterparts and partners’ staff without a

guiding thread and a medium term NECD plan. Several NECD interventions, especially at global level, have

a huge scope, but depend sometimes, for example, on voluntary work of professionals, or the combination

of joint activities of human resources that are involved in other tasks for other initiatives different from

NECD. In this case, a conducive enabling environment supports the sustainability of those initiatives.

Finally, the political cycles are a cause for concern in many countries when new administrations tend to

question automatically the interventions conducted with previous administrations, and change key

counterparts, leading to a loss of institutional memory, a break in the momentum, and the need to move

back a few steps to give the time to the new incumbents to make their own opinion and acquire the

evaluation competences and skills and the time to both UNICEF and counterparts to build trust for a solid

partnership. Actually, the continued presence of UNICEF in nearly all countries, its general appreciation

23 The details of outputs and immediate outcomes for each country case study are presented in Annex 5.


by all parties as an honest broker, and its solid partnerships with a variety of actors, bring an element of

permanence that can facilitate those political transitions.

4.4. GOOD PRACTICES AND LESSONS LEARNT

A number of specific good practices and lessons learnt in UNICEF-supported NECD interventions have

been identified in the case studies.24 25 Among commonly encountered good practices, let us mention

first the capacity to engage a wide variety of international and national actors, even on sensitive issues,

in a respectful and evidence-based way, and progressively build partnerships. For example, at global level,

EVALPARTNERS, EVALSDGs, and GPFE have all been key vehicles to promote partnerships. At country

level, it is clear that national government and non-government actors own the intervention and are

involved at every step of the process. This provides the basis for building large coalitions, focus efforts

and resources, and create a momentum for reform.

Second, successful NECD interventions benefit a lot from work at all three levels (global, regional, and

country) and the interlinkages between those levels. The global level provides a global perspective and an

appreciation of commonalities and differences across regions and there are trickle down effects at

regional and country levels. For example, the spotlights initiative of EVALSDGs had significant trickle-down

effects at the national level. The regional level provides a benchmarking across countries in the region, an

opportunity to learn from more similar countries, and a conduit of communication between the global

and the country levels. The country level provides concrete evidence to the regional and global levels on

the implementation and results of various NECD interventions in different country contexts. Country

Offices have the local knowledge and adapt to the specific country conditions, e.g. to the selection of the

right entry point and seizing opportunities, the identification of champions, and the use of local experts.

The capacity to work at all levels from global to sub-national and relate those levels for cross-fertilization

also adds value as exemplified by the GPFE leading to country parliamentary initiatives or the EAPRO

UNDP-UNICEF NECD intervention leading to more countries of the region jumping on the band wagon of

national M&E system reviews.

Third, NECD interventions suppose competent interveners. Solid in-house expertise in evaluation and

using quality regional and international evaluation experts make a world of difference for credibility and

results as exemplified time and time again in many case studies.

Fourth, the integration of NECD in strategic plans, work plans, evaluation frameworks and M&E systems

goes a long way in making it a central concern, its systematization and institutionalization. An NECD

intervention should not be seen as an isolated project per se, but a key building block for promoting and

solidifying the evaluation function in the whole management cycle, and this at all levels.

24 The details of good practices and lessons learnt for each global or regional case study are presented in Annex 4. 25 The details of good practices and lessons learnt for each country case study are presented in Annex 5.


Fifth, UNICEF corporate culture of “putting things on paper” and documenting experiences in quality

publications and briefings helps keep an institutional memory, disseminate good practices, learn lessons

and adjust, and advocate more credibly based on evidence. For example, EVALSDGs advocacy briefings

have demonstrated to have a wide reach, to be digestible by evaluators, and to be highly influential.

In terms of commonly encountered lessons learnt, a recurrent theme is that the objectives and scope of

any NECD intervention need to be realistic and take into account a diversity of factors: the slowness of

bureaucratic processes in the public sector, the importance of incentives to vanquish vested interests and

resistance to change, the need to accept the legal, political, institutional and cultural dimensions of

evaluation and the complexity of building successful partnerships. For example, the EAPRO UNDP-UNICEF

joint initiative in the Asia-Pacific Region was too optimistic in terms of timeline of output production.

However, the longer time needed to obtain the first outputs was not lost since it helped national

appropriation and dialogue endogenization, thereby building the conditions for more sustainability.

Second, NECD cannot be summed up in a short run intervention; rather it has to be designed and managed

as a medium-term process with a progressive expansion of the scope and sustainability of results through

a programme and a portfolio approach and an institutionalization perspective. The country cases of

Kyrgyzstan and Nigeria demonstrate the value of a multi-faceted and stepwise approach to NECD in which

both evaluation demand and supply are stimulated

Third, success starts at the phase of design and selection of the NECD intervention. The objectives, the

targets and the roadmap need also to clearly defined with a clear Theory of Change. Given resource

constraints, hard choices must be made. Although there is no unique route, most often, the first element

to address in NECD is the enabling environment, then the individual, institutional and ICT dimensions can

be strengthened as clearly identified in the Nigeria case study.

Fourth, there are also some specific lessons depending on the level of capacity-building. At global level, it

is often necessary to “break the silos” and push each partner organization to move beyond its mandate

and rather create a common platform with different perspectives, e.g. the EVALPARTNERS case study. At

institutional level, government ownership is essential for the establishment of an evaluation culture, e.g.

the Philippines case study. At individual level, professionalization of evaluation is a vital component of

NECD, e.g. the Kyrgyzstan, Morocco and Rwanda and Sri-Lanka case studies.

4.5. COMPARATIVE ADVANTAGE OF UNICEF

Notwithstanding the qualitative nature of the data based on perceptions of respondents, there is an

impressive degree of commonality behind the analysis made by respondents on the comparative

advantage of UNICEF, be them from UNICEF staff or from counterpart or partner organizations. Actually,

UNICEF staff tends to be harsher on themselves than their counterparts and partners who all stress the

critical contribution made by UNICEF to NECD. It is obvious to all that UNICEF has been one of the leaders

on NECD among international organizations. It possesses a rich experience in NECD and expertise in NECD

at all levels: global, regional, and country.


Two main comparative advantages rise above the specificities of each NECD intervention26. First, UNICEF

has an uncanny capacity to engage and bring together different types of partners into NECD, i.e.

government, parliamentarians, NGOs, academics, private sector, sub-national actors, international

organizations and other development agencies, etc., and to provide a platform for interaction and

information-sharing among all those interested in M&E. UNICEF is a trusted actor at all levels, recognized

for its competencies in evaluation and its role as an honest broker. Its focus on children is welcome by all,

imposes a cross-cutting approach, and is totally aligned with the 2030 Agenda agreed by all countries.

Second, UNICEF is a relatively strong organization in evaluation able to intervene from global level to sub-

national level through its three-tier organization. It has a capacity to lead or co-lead interventions at all

levels. It is a knowledge-based organization which can easily transfer skills to the locals with high quality

intellectual, ethical and professional standards in M&E. It can count on a set of competent and committed

professionals in evaluation within the organization or, as one counterpart puts it, “a passionate team of

individuals to whom NECD strikes a sensible cord”. It can also mobilize a wide network of professionals,

organizations, and networks around the world who are expert in evaluation and NECD. It has accumulated,

documented and disseminated knowledge on NECD through publications, toolkits, guidelines, etc. By far

and large, UNICEF-supported NECD interventions have been perceived as high performing.

26 Please see Annexes 4 and 5 for the analysis of comparative advantage of UNICEF in each case study.


RECOMMENDATIONS 5


SECTION 5: RECOMMENDATIONS

In conclusion, UNICEF has accumulated good experience in NECD due to its commitment to SDG and child

focus, working at country, regional and global levels, in partnership with key actors and networks in

evaluation. There is a significant demand of continued leadership from UNICEF by interviewed

counterparts and partners in the NECD intervention case studies. UNICEF brings a unique contribution to

NECD through its child-oriented lens (intersectoral, target group of all social policies key to SDGs) and an

ability to work at global, regional, country and subnational levels and engage a variety of partners in solid

partnerships. Work on NECD also brings visibility and credibility to UNICEF and to its Evaluation Office.

The partial evidence gathered through this mapping exercise would argue for a continued and

institutionalized role for UNICEF to support articulated NECD interventions at global, regional, and country

levels with a child focus in the 2030 Agenda to obtain more sizable results in a more sustainable way.

Six specific recommendations are made (relevant UNICEF actors are indicated in parentheses:

Headquarters – HQ, Regional Offices – ROs, Country Offices - COs):

Recommendation 1 (HQ, ROs): Position NECD in relation with the 2030 Agenda in key strategic

documents, including in the forthcoming UNICEF Evaluation Policy and Regional Evaluation Strategies.

This should include among others a clear definition of NECD and its role for the implementation of the

2030 Agenda, guidance on approaches and selection criteria to be used by UNICEF approaches; a clear

description of the linkages between global, regional and country level NECD interventions, and the generic

roles of UNICEF HQ, RO, CO as a team and its partners and counterparts. This would contribute to

institutionalization and facilitate fund mobilization.

Recommendation 2 (HQ, ROs): Continue and intensify its partnership approach in NECD at global and

regional levels and take part in promising NECD and related initiatives initiated by other organizations,

in particular with UN organizations, but also bilateral donors and other actors, i.e. initiatives on data

generation and statistics, Voluntary National Reviews, etc. UNICEF has spearheaded several important

global and regional partnerships (EVALPARTNERS, EVALSDGs, GPFE, EAPRO UNDP-UNICEF joint initiative

in the Asia-Pacific Region) which should be continued with added commitments and resources. Such

partnerships provide a more integrated approach, institutional traction, help mobilize resources and go

through political cycles, lead to more sustainable results, and invite reciprocity. MoUs could be signed

when relevant with key partners, especially at global level.

Recommendation 3 (HQ, ROs): Have HQ and ROs support NECD interventions at lower levels27 where

those high-level actors can make a substantial difference, i.e. help deliver concrete milestones and where

the global or regional perspective is eye-opening, stimulates positive competition among countries,

enables to seize economies of scale. A two-tier approach is suggested combining basic services to all

country offices (sharing practices, promoting South-South and triangular cooperation, documenting case-

27 Lower levels would mean ROs and COs for HQ and Cos for ROs.


studies with support from Evaluation Office) and more intense coaching of selected ROs and COs with

higher expressed interest (demand) and high potential for growth in evaluation capacity (strategic).

Recommendation 4 (HQ, ROs, COs): Consider a variety of possible NECD delivery mechanisms,

combining involvement and leadership of internal staff with performance contracts (LTA type) to regional

and international organizations, whether public, private or non-for-profit, that have a track record of

supporting NECD at individual, institutional, and enabling environment levels.

Recommendation 5 (HQ, ROs, COs): Continue improving the capacity of UNICEF to support NECD in a

sustainable way. This may mean more training and coaching of M&E staff, more systematic sharing of

good practices and lessons learnt from more experienced ROs and COs to the still significant number of

ROs and COs that are not so advanced. Some tools for better selection and monitoring and evaluation of

NECD interventions could also be introduced such as a go/no go grid at country level28 to select NECD

interventions to be supported. Those capacity-building initiatives might be more attractive with financial

incentives such as an innovation fund for new promising NECD interventions to award grants on a

competitive basis among COs.

Recommendation 6 (HQ): Put in place a proper monitoring and documentation system of NECD

interventions on a continued basis. The database of NECD interventions developed within this mapping

exercise can be used as a springboard to develop a web-based information system of the UNICEF-

supported NECD interventions portfolio with decentralized access to a large number of stakeholders and

business intelligence (BI) tools to increase its potential for analysis and its user-friendliness. This data base

and supporting BI tools could be made available to other international organizations to share costs and

benefit from economies of scale as well as obtain a more global perspective on NECD interventions.

28 An example of possible go/no go grid at country level is presented in Annex 6.


ANNEXES


ANNEX 1 – ORGANIZATION OF THE MAPPING EXERCISE

a. Project Manager

• Mrs. Ada Ocampo

Senior Evaluation Specialist

Evaluation Office

UNICEF New York

[email protected]

b. Reference Group

Name Position E-mail

Tetyana Nikitina UNICEF Sri Lanka [email protected]

Hammad Masood UNICEF Angola [email protected]

Justus Kamwesigye UNICEF ESARO [email protected]

Riccardo Polastro UNICEF EAPRO [email protected]

Fabio Sabatini Evaluation Office [email protected]

Ziad Moussa EVALPARTNERS Co-chair [email protected]

Dorothy Lucks EVALSDGs Co-chair [email protected]

[email protected]

Benoit Gauthier IOCE [email protected]

Javier Guarnizo UNIDO [email protected]

Brook Boyer UNITAR [email protected]

c. Consultant Team

• Dr. Frederic Martin

Team Leader

[email protected]

• Mrs. Ana Maria Fernandez


[email protected]

• Mrs. Giovanna Donado


[email protected]

• Mr. Jacques Lefevre

Senior IT Specialist

[email protected]

mailto:[email protected]





d. List of persons interviewed during the inception phase

Name Position E-mail

Fabio Sabatini Senior Advisor, Research and Evaluation Office [email protected]

Koorosh Raffii Senior Evaluation Specialist, Evaluation Office [email protected]

Isa Achoba Associate Director Field Results Group (FRG) [email protected]

Laura Olsen Evaluation Specialist, Evaluation Office [email protected]

Lovemore

Mhuriyengwe

Knowledge Management Specialist, Evaluation

Office

[email protected]

Matthew Varghese Sr. Evaluation Specialist, Evaluation Office [email protected]


ANNEX 2: PROPOSED THEORY OF CHANGE FOR NECD

A generic Theory of Change (TOC) is presented as contextual information for this mapping exercise in

Figures 2.1, 2.2, and 2.3, respectively for enabling environment, institutional, and individual dimensions

of capacity development


Figure A2.1: Proposed Theory of Change for NECD at Enabling Environment Level

Assumptions/Risk factors

UNICEF budget allocations for enabling environment NECD Trained and experienced UNICEF staff in enabling

environment NECD Committed UNICEF staff

Working strategic alliances and partnerships

Political stability Political and technical leadership in key institutions and partners

Policymaking coordination across institutions Enabling international environment

Limited resistance to change in key institutions

Impacts

Enabling environment

level

Final beneficiaries

Activities

Outputs Immediate Outcomes

Final Outcomes

Technical and financial support and advocacy with right selection of:

• Strategic orientations

• Target groups (institutions, committees, fora, etc.)

• Support areas corresponding to critical NECD gaps and UNICEF comparative advantage

• Cross cutting issues related to child rights such as human rights, gender equality, and equity and SDGs

• Strategic alliances and partnerships

• Support institutions and experts

• Support systems and tools as well as approach

• Duration and timing

• Sequencing and articulation with other NECD activities

• Adequate funding of NECD activities

• Adequate management of NECD activities (planning, support during execution, monitoring and adjustment)

Reduced critical capacity gaps in

terms of formulation and

application of evaluation policies,

institutional processes, evaluation

capabilities, and partnerships, to

promote the implementation of

child rights and SDGs at global,

regional and national levels

Contribution to an increased

quantity, variety, and quality of

evaluations, and the dissemination of an evaluation

culture in policies, programs, and

projects to promote the

implementation of child rights and

the achievement of SDGs at global,

regional, and national levels

Contribution to the

achievement of SDGs at global,

regional and national levels

Contribution to an

increased use of evaluations to formulate

and revise policies,

programs and projects

related to child rights and SDGs at

global, regional, and

national levels

Contribution to the

improvement of the situation of

children and women at

global, regional and national

levels


Figure A2.2: Proposed Theory of Change for NECD at Institutional Target Group Level

Impacts

Institutional level

Final beneficiaries

Activities


Final Outcomes


• Strategic orientations

• Target groups (institutions)

• Support areas corresponding to critical NECD gaps in key institutions and to comparative advantage of UNICEF

• Cross cutting issues related to child rights such as human rights, gender equality, equity, and SDGs

• Partners

• Support institutions and experts

• Support systems and tools as well as approach




• Adequate management of NECD activities (planning, support during execution, monitoring and adjustment)

Reduced critical NECD gaps of target institutions in terms

of strategic orientations, institutional

arrangements, technical systems for

the evaluation of policies, programs,

and projects to promote the

implementation of child rights and the

achievement of SDGs at global, regional, and

national levels


UNICEF budget allocations for institutional NECD Trained and experienced UNICEF staff in institutional NECD

Committed UNICEF staff Working partnerships

Existence and endurance of champions of change in key target institutions

Limited resistance to change in key target institutions UNICEF leadership

Existence and endurance of champions of change in key target institutions


Contribution to the

improvement of the situation of

children and women at

global, regional and national

levels

Contribution to an increased use of evaluations by key institutions

in policies, programs and

projects related to child rights and SDGs to

improve client orientation,

service delivery processes, HR

management, FR management, Information

systems management


quantity, variety, and quality of

evaluations, and the dissemination of an evaluation

culture in policies,

programs, and projects to

promote the implementation

of child rights and the achievement of SDGs at global,


Contribution to the

achievement of SDGs at

global, regional and

national levels


Figure A2.3: Proposed Theory of Change for NECD at Individual Target Group29 Level

29 The individual target group level refers to key staff in key institutions and to comparative advantage of UNICEF. They can be working for national, regional, or global institutions, including UNICEF

Impacts Activities


Individual level

Institutional level

Final beneficiaries

Final Outcomes


• Target groups (institutions, participants)

• Sensitization and training areas corresponding to critical evaluation capacity gaps of key staff in key institutions and to comparative advantage of UNICEF

• Cross cutting issues related to child rights such as human rights, gender equality, equity, and SDGs

• Partners

• Sensitization and training institutions and trainers

• Sensitization and training material and approach

• Venue and logistics




• Adequate management of NECD activities (planning, support during execution, monitoring)

Reduced critical capacity gaps in

terms of knowledge, skills,

attitudes, experience and

behaviour of key staff in key

institutions for the evaluation of

policies, programs, and projects to

promote the implementation of

child rights and the achievement of SDGs at global,


Contribution to the improvement of the situation of children and

women at global, regional and

national levels


Contribution to an increased use of evaluations by key institutions in

policies, programs and

projects related to child rights and SDGs to

improve client orientation,

service delivery processes, HR

management, FR management, Information

systems management

UNICEF budget allocations for individual NECD Trained and experienced UNICEF staff in individual NECD

Committed UNICEF staff Working partnerships

Limited capacitated target group rotation Managers in key national institutions ready to use

capacitated individuals Limited resistance to change in key institutions

UNCEF leadership

Existence and endurance of champions of change in key institutions



quantity, variety, and

quality of evaluations,

and the dissemination

of an evaluation culture in policies,

programs, and projects to

promote the implementation

of child rights and the

achievement of SDGs at global, regional, and

national levels

Contribution to the

achievement of SDGs at

global, regional and

national levels


ANNEX 3 – DATA COLLECTION INSTRUMENTS

There are two data collection instruments:

1. The Online survey questionnaire;

2. The In-depth Interview questionnaire.

The basic questionnaire was the same, but there were some differences. However, the online survey,

questionnaire which aimed at identifying NECD interventions, was administered to UNICEF M&E staff at

global, regional and country levels who could identify and document up to three NECD interventions.

There were very few open questions in this questionnaire. The in-depth interview questionnaire, which

aimed at documenting specific NECD interventions considered as case studies, was administered partly

on line, partly in an interview to key actors from both UNICEF, partners and/or counterparts.

Key sections of the basic questionnaire are presented below.

SECTION 1: CHARACTERIZATION OF UNICEF-SUPPORTED NECD INTERVENTION X

NECD X

Name

Responsible agency or unit

Location

Beneficiaries

Partners (if any)

Main objectives

Actual start year

Actual or planned end year

Indicative duration (in months)

Indicative budget (USD000’)

Main outputs

Main immediate outcomes

Levels of NECD intervention NECD X

Global

Regional

National

Sector

Sub-national


Type of country NECD X

Middle income country

Low income country

Fragile

Other (specify):

Work domain in which the NECD took place NECD X

Humanitarian

Development

Target group of NECD intervention NECD X

Government

Parliament

Private sector

NGOs

CSOs

Academia

Other (specify):

Dimension of NECD NECD X


Institutional level

Individual level

Objectives of NECD NECD X

Contribute to the development of national evaluation policies or frameworks

Support the inclusion of evaluation in constitutions or other legal frameworks

Contribute to crafting national or sectoral evaluation plans and agendas

Support the establishment of evaluation bodies within ministries or other national

institutions

Build up partnerships to conduct NECD

Support the conceptualization and implementation of country-led evaluations

Develop/revise curricular in evaluation in higher/vocational education institutions

Provide training and professional development opportunities on evaluation

Develop and conduct awareness raising and advocacy activities for NECD

Conduct advocacy for budgetary allocations for evaluation

Raise funding for NECD

Other (specify):


Explicit mention of crosscutting dimensions in NECD NECD X

Human rights

Equity

Gender equality

SDGs

Other (specify):

UNICEF role NECD X

Undertaken independently by UNICEF

Undertaken with partner organizations with UNICEF as a leader or coordinator

Undertaken with partner organizations with UNICEF as a supporting role

Partner(s) of UNICEF in NECD intervention NECD X

International organizations

International fora

Government

Parliament

Private sector

NGOs

CSOs

Academia

Other (specify)

Please add any additional information you see relevant for the sake of characterization of NECD

intervention X

SECTION 2: PERCEIVED PERFORMANCE OF UNICEF-SUPPORTED NECD INTERVENTION X

Extent to which NECD intervention X is in line with global reference documents

Possible grades and meaning:

0 No alignment

1 Partial and indirect alignment

2 Partial, but direct alignment

3 Full and direct alignment

NA Not able to answer


Please indicate the relevant grade for each NECD intervention.

Global Reference Document NECD X

UNICEF Evaluation Policy

UNICEF Strategic Plan, 2014-2017

GA resolution on National Evaluation Capacity Development

GA resolution on the Agenda 2030

Global Evaluation Agenda

UNEG Norms and Standards

Other (specify):

Performance criteria of UNICEF-supported NECD intervention X

Possible grades and meaning:

0 Low

1 Somewhat

2 To a significant extent

3 High

NA Not able to answer

Please assess each NECD intervention according to performance criteria indicated below.

Performance criteria NECD X

Relevance to meet NECD critical gaps

Effectiveness

Efficiency

Outputs

Immediate outcomes

Sustainability of increased capacity built by NECD intervention

Why? (the reasons behind the observed performance, for example analysis of factors supporting or

constraining successful outcomes; the context for interventions, including opportunities and risks, etc.)

UNICEF comparative advantage in terms of NECD

Please list major comparative advantages of UNICEF for key UNICEF NECD interventions. Please prioritize

the comparative advantages, with 1 being the most important one.

1.

2.

3.


Good practices and lessons learned on NECD

Please list major good practices from key UNICEF NECD interventions. If the answer differs significantly

among the listed NECD interventions you selected, please say how.

1.

2.

3.

Please list major lessons learnt from key UNICEF NECD interventions. If the answer differs significantly

among the listed NECD interventions you selected, please say how.

1.

2.

3.

Recommendations for future NECD interventions

Please indicate a set of realistic, affordable and practical recommendations for NECD support. If the

recommendation is made to a specific target group, please indicate the relevant target groups.

Recommendation

1.

2.

3.

Please indicate below any other suggestions or comments useful for this mapping exercise on UNICEF’s

support to national evaluation capacity development (NECD)


ANNEX 4 – CASE STUDIES OF NECD INTERVENTIONS AT GLOBAL AND REGIONAL LEVELS

NECD INTERVENTION: EVALPARTNERS

Characterization of the NECD intervention


(mechanism responsible)

UNEG and IOCE

Location Global (formally registered in Ottawa, Canada)

Beneficiaries

Governments, parliamentarians, Voluntary Organizations of

Professionals in Evaluation (VOPEs), UN organizations, bilateral

organizations.

Partners 60 partners who are at the same time the beneficiaries listed above.

Main objectives 1. Strengthen national evaluation capacities

2. Improve civil society evaluation capacities

Actual start year 2012

Performance of the NECD intervention

This NECD intervention has demonstrated good performance overall. Its relevance is obvious since

EVALPARTNERS is a global partnership to strengthen national evaluation capacities. It has been largely

effective in terms of producing outputs. It has led to several publications such as advocacy briefing notes,

organization of conferences, fora, peer to peer exchanges, e- learning, e-learning platform and initiatives,

partnerships. EVALPARTNERS has influenced several initiatives: together with UNEG, it advocated for and

influenced the GA Resolution on National Evaluation Capacity Building. It worked closely with UNEG to

advocate for the incorporation of Evaluation in the Agenda 2030. It launched the Global Evaluation

Agenda 2016-2020 after a consultative multi-stakeholder process. It helped establish networks supporting

two streams: EVALSDGs and the Global Parliamentarians Forum for Evaluation (GPFE), and through those

networks, EVALPARTNERS has engaged parliamentarians, governments and other key actors in the

promotion of evaluation as an important element for good governance. A growing number of parliaments

have incorporated evaluation in national legislation or constitutions. A number of National Evaluation

Policies (NEP) have been elaborated and adopted at country level. Nowadays, EVALPARTNERS acts as an

umbrella for several global networks: EVALGENDER+, EVALSDGs, the Global Parliamentarians Forum for

Evaluation (GPFE), EvalIndigenous and EvalYouth. EVALPARTNERS has clearly strengthened the enabling

environment for evaluation by increasing the demand and the use of evaluation and demonstrated the

power of partnerships for influencing global agendas.


Its efficiency has been assessed as relatively good, although some respondents said it could be better

given the amount of resources invested. One complicating factor in the analysis is that it is hard to

differentiate between the inputs and the outputs of a given initiative supported by EVALPARTNERS and

the overall inputs and outputs of EVALPARTNERS per se.

UNICEF has played a major role in EVALPARTNERS, among others the promotion of partnerships. UNICEF

created EVALPARTNERS and seats at the management group. This corresponds to UNICEF strategy and it

also is consistent with the mandate from UNEG to push Strategic Objective 4 on Partnerships. Enabling

factors have been the good buy in and support from UNICEF management on the importance in ECD and

resource mobilization for NECD, as well as key contributions of several UNICEF professionals.

Extent to which the NECD intervention is in line with important reference documents

There is a large alignment of the NECD intervention with key reference documents such as the UNICEF

Evaluation Policy, the GA resolution on National Evaluation Capacity Development, the Global Evaluation

Agenda, the UNICEF Strategic Plan 2014-2017, and UNEG Norms and Standards. This is to be expected

from an NECD intervention associating key global stakeholders.

Extent to which the NECD intervention has given attention to human rights, equity, gender equality,

and SDGs

The NECD intervention considers a variety of cross-cutting dimensions such as human rights, equity,

gender equality, SDGs, and indigenous rights.

Good practices on NECD

Good practices noted in this NECD intervention include: i) a corporate culture of “putting things on paper”,

i.e. publications such as books, policy briefs. This practice can be useful for other NECD actors, in particular

several CSOs which tend to be less organized in their documentation of their activities, processes, and

results, ii) a capacity to touch upon and bring to the policymakers’ table a variety of complex, sensitive,

and cross-cutting issues such as employment, youth protection, gender-based violence, and, iii) a

capability to work at all levels and relate them.

Lessons learnt on NECD

Several lessons can be learnt from this NECD intervention. At enabling environment level, EVALPARTNERS

has a key role to play to contribute to “breaking the silos” and pushing each organization to move beyond

its mandate and rather create a common platform with different perspectives. EVALPARTNERS could be

more involved in global multi-stakeholder partnerships where lies its main comparative advantage and

less at national level to avoid spreading out its resources on too many activities. Among global issues it

could push further, alignment with the Agenda 2030 is critical to move the evaluation agenda forward.

More work could also be done to push evaluation in humanitarian and fragile contexts. At institutional


level, government ownership is essential for the establishment of an evaluation culture. At individual

level, professionalization of evaluation is a vital component of NECD.

UNICEF comparative advantage in NECD

The major comparative advantage of UNICEF in term of NECD is its capacity to bring together actors. First,

UNICEF is a trusted actor at all levels, recognized for its competencies in evaluation and its role as an

honest broker. It does engage with a variety of actors: government, private sector, civil society and VOPES,

at all levels, from global level to regional, national and sub-national levels thanks to its wide network of

regional and country offices. It can count on a set of competent and committed professionals in

evaluation. As one counterpart puts it, “a passionate team of individuals to whom NECD strikes a sensible

cord”.

Recommendations to UNICEF

Recommendations are to: i) continue its implication and allocate more resources; ii) position NECD in

relation with the 2030 Agenda in the forthcoming UNICEF Evaluation Policy for more institutionalization

and sustainability; iii) push for the inclusion of NECD in Regional Evaluation Strategies; iv) facilitate peer-

to-peer learning on NECD; and v) promote the institutionalization of all good practices.

Final conclusion

EVALPARTNERS has proven to be a very good NECD intervention to foster global NECD initiatives and

partnerships for a more coordinated NECD. It has offered a positive ecosystem and institutional

framework for several networks and NECD interventions. Efficiency and sustainability could be somewhat

improved. It could focus more at the global level in its advocacy efforts, and at the same time, promote

and support enabling environments for evaluation at regional and national levels especially, especially in

the context of the 2030 Agenda.


NECD INTERVENTION: EVALSDGS


Responsible agency or unit IOCE and UNICEF/Evaluation Office

Location Global

Beneficiaries Parliamentarians, policy makers, UN, international organization,

academia, VOPEs, National Governments

Partners Global network under the umbrella of EVALPARTNERS

Main objective

1. Promote evaluation activities around the SDGs

2. Orient and support the review and follow-up process on the SDG

2030 Agenda by adding value through evaluation.



This NECD intervention has demonstrated good performance, especially if one takes into account its

limited budget for a huge mandate. It is obvious that relevance is high given the importance of evaluation

and of SDGs. Let us note that the focus is not only on the evaluation of the achievement of SDGs per se,

but on supporting the demand for and the supply of evaluations of policies and programs that contribute

to SDGs, and even further than that contribute to the awareness about the evaluation function and the

dissemination of an evaluation culture.

It is also clear that this intervention has been effective and efficient with a wide variety of outputs

produced for little money: i) several important publications, including guidance documents and eight

briefing advocacy papers launched on evaluation and SDGs with various partners e.g. IEG/World Bank and

UNDP. The Briefing papers are a joint venture between IIED and EVALSDGs and are among the most

downloaded papers from the IIED website; ii) identification of spotlight countries for the preparation

towards the evaluation of the SDGs, provision of technical support to those countries, and documentation;

iii) communication and learning platform with monthly webinars on country case studies; iv) organization

of regional consultations in partnership with the Global Parliamentarians Forum for Evaluation with

orientation provided to countries to develop plans to establish national evaluation policies in line with the

agenda 2030; v) advocacy for the establishment of frameworks for SDGs evaluation and how to influence

the Voluntary National Reviews (VNRs) which are becoming an important part of the SDG evaluation

process.

It is way too early to talk about final outcomes, however EVALSDGs has already demonstrated its potential

of value added to the Agenda 2030 through the orientation on and the promotion of Evaluation, its


platform for dialogue, advocacy and knowledge generation in relation to evaluation and the SDGs and its

contribution to a better appreciation by stakeholders of inter-linkages and interaction between initiatives

to strengthen evaluation and implement and review the SDGs.

The challenge is sustainability. Even if the networking capacity, the voluntary work and the support from

UNICEF are appreciated, the budget is low (USD 30.000-40.000 annually) which reduces the capacity to

conduct more activities while the agenda is large for a small group of voluntary people, which means the

choice of activities needs to strategic.


This NECD intervention is aligned with important global strategic documents and policy guidelines. This is

to be expected given its focus at global enabling environment level and the importance of SDGs for the

UN organizations, including UNICEF.


and SDGs

The NECD intervention considers human rights, equity, gender equality, SDGs and environment as cross-

cutting dimensions.


Good practices on NECD are: i) advocacy briefings which have demonstrated to have a wide reach, are

digestible by evaluators, and to be highly influential; ii) the partnership of EVALSDGs with the GPFE

enhances results at the national level; iii) the spotlights initiative which had trickle down effects at the

national level.


Lessons learnt from this NECD point to: i) the need to work more in establishing clear linkages between

global, regional and national levels with mutually beneficial effects. The global level provides a global

perspective and an appreciation of commonalities and differences across regions and there are trickle

down effects at regional and country levels. The regional level provides a benchmarking across countries

in the region, an opportunity to learn from more similar countries, and a conduit of communication

between the global and the country levels. The country level provides concrete evidence to the regional

and global levels on the implementation and results of various NECD interventions in different country

contexts; ii) the importance of human and financial resources to carry out interventions at global level.

The scope of outputs and outcomes is obviously limited by the resource constraints.



UNICEF comparative advantage in terms of NECD are: i) a strong evaluation practice within the

organization. The technical contents of evaluations conducted at UNICEF follow international standards;

ii) its focus on children. The “No one left behind” strategic goal is clearly embedded in the organization

which focuses on the marginalized with an integrated perspective; iii) its capacity to lead or co-lead global

initiatives; and, iv) its wide network of M&E professionals at the global level. UNICEF has provided

invaluable support to this NECD intervention, mainly in the form of professional time and facilities.


Recommendations made to UNICEF are to: i) incorporate NECD and its relation with the 2030 Agenda in

the forthcoming UNICEF Evaluation Policy; ii) assign annual resources to NECD with a focus on reinforcing

capacities for the evaluation of the SDGs; iii) consider bringing child-focus evaluation to the EVALSDGs

initiative by developing work around thematic areas such as how to do you evaluate SDGs from an

integrated point of view focusing on the nexus around children.

Final conclusion

This recent NECD intervention is dealing with a major issue not only for UNICEF, but also for the UN system

and all other development actors. With limited budget, it has contributed significant and useful outputs,

including advocacy briefing papers, identification of spotlight countries for the preparation towards the

evaluation of the SDGs, monthly webinars on country case studies, organization of regional consultations,

and advocacy for the establishment of frameworks for SDGs evaluation and how to influence the

Voluntary National Reviews (VNRs). One main strength of the approach has been to work in partnerships.

UNICEF has provided limited financial support, but critical indirect support in terms of logistics and direct

institutional support.


NECD INTERVENTION: GLOBAL PARLIAMENTARIANS FORUM FOR EVALUATION (GPFE)

Note: the GPFE was initially known as the Parliamentarians’ Forum for Development Evaluation (PFDE).


Responsible agency or

unit

Network under EVALPARTNERS headed by a Steering Committee

supported by a Secretariat

Location Global (current Chair is in Colombo, Sri Lanka)

Beneficiaries Parliaments, parliamentarians, governments, and civil society

Partners

Parliaments, EVALPARTNERS, EVALSDGs, EvalGender, World Bank IEG,

African Development Bank, CLEAR Centres, UNICEF and other UN

agencies for specific activities.

Main objectives

1. Strengthen the demand and the use of evaluation by

parliamentarians.

2. Create enabling environments for nationally owned, transparent,

systematic evaluation processes.

3. Promote national evaluation policies and systems.

Actual start year • 2013 for the first ever Regional Parliamentarians Forum for South Asia

launched in in Colombo.

• 2015 for the launch of the Global Movement the main event

celebrating EvalYear 2015


This NECD intervention enjoys a high performance. It is obviously relevant given the key role of the

legislative branch in approving the laws, policies, and budget proposed by the executive branch in

democratic countries. Parliamentarians are the ideal actors to perform an important role to ensure budget

allocations are justified by evaluated development results, to trigger the evaluation agenda, to request

specific evaluations and to promote a culture of evaluation at national level. Through capacity building,

knowledge sharing and advocacy, GPFE empowers parliamentarians to deliver evidence-based policies

and thus contributes to good governance, sustainable development and social equity.

The GPFE has been effective in producing a number of outputs, including: 1) in terms of advocacy

activities, the GPFE Launching New Website & Social Media- Parliament of Nepal on 25th November

201530, a video campaign inviting parliamentarians around the globe to create video messages on the

30 https://www.europeanevaluation.org/forums/general-discussion/global-parliamentarians-forum-evaluation-

gpfe-launching-new-website-social and https://globalparliamentarianforum.wordpress.com/

https://www.europeanevaluation.org/forums/general-discussion/global-parliamentarians-forum-evaluation-gpfe-launching-new-website-social

https://www.europeanevaluation.org/forums/general-discussion/global-parliamentarians-forum-evaluation-gpfe-launching-new-website-social

https://globalparliamentarianforum.wordpress.com/


importance of National Evaluation Policies (NEP) which resulted in 38 videos available in multiple

languages, advocacy products e.g. “10 reasons why parliamentarians should be involved in evaluation”

and the “Meet a parliamentarian” webinar series; 2) in terms of creating and sharing knowledge: mapping

on NEPs, regional consultations and partnerships on NEPs, case studies on National M&E systems with

gender and equity focused lens.

There are also already visible outcomes in terms of increased awareness of parliamentarians on the

importance of evaluation which led to the establishment of regional parliamentarians’ fora and changes

in the legal framework of countries to take into account evaluation. Between 2014-2015, regional

parliamentarians’ fora were created in Africa, East Asia, Latin America and MENA regions. The first ever

Parliamentarians’ Forum for Development Evaluation (PFDE) was established in South Asia in early 2013.

The African Parliamentarians Network on Development Evaluation (APNODE) was initiated at the AfrEA

conference held in Yaoundé, Cameroon in March 2014 after one year of initiating PFDE. APNODE is hosted

and supported by the African Development Bank. Morocco and Ivory Coast have included evaluation as a

necessity to implement legislation and national systems in their constitutions. In Nepal, evaluation is

included directly in the legislation. In Tunisia, it is included in the parliament regulation. Other Parliaments

have taken specific actions to consider evaluation in key domains: Pakistan’s Parliament established an

SDG Secretariat to help parliamentarians effectively oversee SDG progress, address the legislative gaps

and ensure their constituents’ rights. In Trinidad and Tobago, the Parliament established a new Joint

Select Committee on the Environment and Sustainable Development. In Zambia, MPs informed a SDGs

caucus. In Sri Lanka, the Parliament established a Select Committee on the SDGs. In Finland, MPs were

consulted during the preparation of the first Voluntary National Review, chair the Finnish Development

Policy Committee that monitors and assesses implementation of Finland’s international commitments,

and they are widely represented in the National Commission of Sustainable Development that governs

implementation of the SDGs. In the United Kingdom, a Parliament Committee has conducted a critical

inquiry that encouraged a shift in government approach to addressing the SDGs. Parliamentarians have

also spearheaded major evaluation events in national and regional parliaments including in Nepal, Sri

Lanka, Tunisia, Kyrgyz Republic and in the Latin American parliament called “Parlatino”. There are

parliamentarians’ groups on evaluation in the Middle East and North Africa, East Asia, Latin America and

Eurasia. National parliamentarians’ fora on evaluation have been launched in Nepal, Uganda, Kenya and

Sri Lanka. NEPS have been elaborated and validated in Parliament in a number of countries. It is too early

to talk about final outcomes, but immediate outcomes are definitely there.

In terms of sustainability, the constitutional nature of the Parliament, the interest and involvement of

MPS, the support of EVALPARTNERs and from UNICEF since the beginning are contributing factors.

However, other factors can negate those factors such as the high mobility of politicians, government

cycles, political crises, and the tendency in some countries towards more autocracy and populism. Also,

the GPFE has been so far a resounding success in developing countries, but there is much less involvement

of Parliaments from industrialized countries.



This NECD intervention is aligned with important global strategic documents and policy guidelines. This is

to be expected given its focus at global enabling environment level.


and SDGs

This NECD intervention considers human rights, equity, gender equality, and SDGs as cross-cutting

dimensions in the intervention.


Good practices in NECD include: i) the importance of having a strong and committed leader of the forum

to be proactive in terms of activities and create a momentum which brings news MPs and new countries

on board; ii) the establishment of wide partnerships to maximize synergies, resources, and in the end

results, iii) the respect of country-specific ways of working, iv) the knowledge and experience exchanges

among MPs from various countries.


The main lesson learnt from this NECD intervention is that the evaluation community still needs to learn

about the political dimension of evaluation. Some evaluators find it difficult to understand the role of

parliamentarians as they see evaluation only as a technical exercise outside of its legal, political,

institutional and cultural context.


UNICEF has supported the PFDE since its inception. It supports the Steering Committee of the GPFE as

well as the Secretariat. Currently it takes part in the organizing committee of EvalColombo 2018, the first

ever Global Parliamentarians event to take place in September 2018.

The comparative advantages of UNICEF in term of NECD come from its expertise in evaluation, its wide

network of country offices with channels and access to parliamentarians and its recognition as a non-

partisan honest broker working with MPS from the political parties in power as well as from the opposition

parties. UNICEF has also a capacity to identify “champions” and “wining situations” as a locally present,

but neutral actor in the countries.



Parliamentarians are responsible for making national government accountable and should therefore be

strong advocates for using evaluation in policymaking. UNICEF should therefore continue to play a key

role to support this intervention, in particular to: i) further develop the parliamentarians’ awareness of

the importance of NECD and the need to allocate resources to NECD and expose them to different NECD

strategies; ii) involve parliamentarians in the work UNICEF does to influence legislation for children, be it

the revision of existing policies or the elaboration of new policies; iii) further develop parliamentarians’

capacities to assess programs and policies in the context of the 2030 Agenda. The support of UNICEF

should be opportunistic and be adapted to the demands and capacities of parliamentarians’ fora in

different countries.

Final conclusion

Created as a network of EVALPARTNERS, the PDFE which became the GPFE is a well-respected forum.

Started in a few countries in Asia, it has since mushroomed in Asia, but also in Africa and Latin America.

Parliamentarians are a key stakeholder in democratic systems and can play a major role for evaluation as

well as benefit greatly from evaluation. This NECD intervention has rapidly created products and

generated outcomes. UNICEF has played a key role from the start and should continue its support in a

context, making it tailor-made to the situation in each country and adapted to emerging evaluation issues.


NECD INTERVENTION: EAPRO UNDP-UNICEF JOINT INITIATIVE AND COUNTRY CASE STUDIES IN THE ASIA-PACIFIC

REGION



(mechanism responsible)

UNDP and UNICEF

Location Asia and the Pacific

Beneficiaries

Governments of Indonesia, Malaysia, Thailand, and Sri Lanka in

2017; Mongolia, the Philippines and others in 2018

Partners Asian Development Bank and UNWOMAN in peer review

Main objectives

1. Generate knowledge to guide NECD for 2030 Agenda

2. Identify successes and lessons in terms of national evaluation

systems

3. Foster peer learning and provide global and regional NECD

guidance through the production of readiness assessments at

country level


Actual or planned end year 2019

Indicative duration (in months) 36


The initiative originated from Multiple regional and global fora focusing on national evaluation capacity

development including: NEC commitments and NEC Bangkok declaration, Parliamentarians Forum for

Evaluation by EVALPARTNERS in partnership with UN agencies, Global Evaluation Agenda developed by

EVALPARTNERS, other UN agencies, and a regional training workshop on “Developing National Evaluation

Capacities to Support the Implementation of the 2030 Agenda”. All actors perceived it did not make sense

to approach NECD in an isolated way per institution. All institutions shared the diagnostic and the need

to reinforce data, monitoring, and performance budgeting. This common diagnostic and objectives

facilitated the design of a consistent regional intervention by UNDP and UNICEF which is not easy given

the number of actors involved and the differences among country situations. Other facilitating factors

included the buy-in from country counterparts, the in-country liaison between UNICEF CO and

government professionals, and the interest of other agencies such as ADB.


Performance of this NECD intervention is globally high, recognizing that this is an ongoing intervention, so

this is only a preliminary assessment. In term of relevance, the NECD intervention is important to identify

and then reduce critical gaps to a proper evaluation function contributing to good governance. The

effectiveness to meet its objectives is high even though the process took much longer than expected (1

case study was completed out of the 4 planned in Year 1). The first two country diagnostics provide useful

information on (i) What M&E systems are in place to monitor, review and evaluate implementation of

2030 Agenda? (ii) How do evaluations inform adjustments to strategies, resource allocation and policies?

Is this changing in light of 2030 Agenda? (iii) What evaluation methods/ tools are government/ other

stakeholders using to evaluate policy choices to support integrated approaches? What equity-based

evaluation methods/tools are being used? Could they be scaled up to monitor commitment of ‘leaving no

one behind’? Those reports also make useful recommendations to strengthen national evaluation

capacity for the SDGs. Efficiency in terms of value for money is also high with an interesting combination

of work done by local consultants and guidance from regional and international recognized experts.

Although it is too early to tell about immediate and final outcomes, one can already anticipate the benefit

of having a regional synthesis, the value of peer to peer exchanges, and the ripple effect of this

intervention on other countries in the region who want to jump on the wagon and will position

participating countries in a position to evaluate the SDGs.


There is full and direct alignment of the NECD intervention with relevant documents such as UNICEF

Evaluation Policy, UNICEF Strategic Plan (2014-2017), the GA resolution on National Evaluation Capacity

Development, the Global Evaluation Agenda and UNEG Norms and Standards. This reflects the fact that

UNICEF-UNDP project managers know about those reference documents, understand them, and are able

to implement them in concrete NECD interventions.


and SDGs

Even if the NECD intervention focuses on the generation of knowledge to guide NECD in the perspective

of the 2030 Agenda and the identification of successes and lessons in terms of national evaluation

systems, it considers human rights, equity, gender equality, and SDGs as cross-cutting dimensions in the

NECD intervention.


Good practices include: i) the case study process based on the framework of EvalAgenda 2030 from

EVALPARTNERS; ii) the productive collaboration between UNDP which enjoys strong linkages with

national governments and UNICEF which also enjoys linkages with governments but also with other

stakeholders, strengthens the UN message and credibility to the governments; iii) the integration of NECD

as part of UNICEF CO work plans; iv) the opportunities of entry points related to the country case studies


and other ongoing UNICEF initiatives (e.g. Thailand evaluation policy); v) the hiring of national consultants

to develop national case studies; vi) the documentation of the case study31


Lessons learnt so far are that: i) the scope and timeframe of the NECD intervention were not consistent

with resources available; ii) follow up on existing case studies and elaborating more case studies is very

important. NECD should not be conceived as a one-off activity, but as a stepwise process; iii) it is important

to make NECD part of the evaluation work plan; iv) While work on MDGs used to be done in silos, SDGs

require interoperability of systems and so governments want an integrated approach.


The comparative advantages of UNICEF in terms of NECD are: i) a stronger focus on NECD than other

international organizations; ii) its presence in the countries, and not only in the capital city, but also in the

regions. UNICEF COs have national linkages and a level of credibility at country level which gets the

message of evaluation on the agenda. This provides an in-depth perspective for the NECD intervention;

iii) its technical capability in evaluation and experience in supporting the elaboration of evaluation policies

and M&E systems. Evaluations conducted by UNICEF are quality products that are relevant and practical

since they produce recommendations at programme level; and iv) its dedicated M&E staff in UNICEF COs.


Recommendations are that UNICEF HQ and ROs engage in key global and regional strategic partnerships

on NECD such as this NECD intervention and do so with a medium term perspective while UNICEF COs

should continue to i) support evidence generation activities of Government thought a variety of surveys

(e.g. LICS, MICS and HICS) and evaluations as CP priorities; ii) identify capacity gaps that prevent from

acting at policy or program level; iii) work with partners that are involved in planning and in budgeting, so

that the use of evaluations for planning and budget be considered and iv) reinforce their sub-national

focus.

Final conclusion

This regional NECD intervention clearly demonstrates the benefits of an integrated approach: first

integrated in terms of partners with UNDP and UNICEF working together and progressively bringing in

other partners such as ADB, WB, JICA, Australia; integrated also in terms of approach to be consistent

with SDGs which require interoperability of systems, a multisectoral perspective and a clear roadmap with

31Polastro, R. and M. Prokop (2017). Review of National Evaluation Systems and Capacities for Evaluating Progress

Towards the Sustainable Development Goals - Emerging Lessons Learnt from the Asia Pacific Region, Paper presented at the NEC Conference, Istanbul


actions, labour division, and coordination; and finally integrated in terms of results which must go beyond

fragmented outputs such as meetings, support to VOPEs, and publications to move toward a clear Theory

of Change of UNICEF-supported NECD interventions.


ANNEX 5 – CASE STUDIES AT COUNTRY LEVEL

CASE STUDY OF NECD INTERVENTION: SOUTH TO SOUTH COOPERATION BETWEEN THE M&E UNIT IN THE COLOMBIAN

AND ECUADORIAN PLANNING MINISTRIES


Responsible agency or unit UNICEF Ecuador (and participation of UNICEF Colombia)

Location Ecuador and Colombia

Beneficiaries

Secretaria Nacional de Planificación y Desarrollo (National Planning

and Development Ministry)

Consejo Nacional de Planificación (National Planning Commission)

Partners (if any) M&E units in the Colombian and Ecuadorian Planning Ministries

Main objective

Introduce the M&E Unit (Planning Ministry Ecuador) to good practices

from Colombia on how to develop and implement an evaluation

agenda



Indicative duration (in

months)

3


The NECD intervention was relevant since it was a quick way to move the evaluation agenda. Colombia

was the right kind of country to go forward for Ecuador, sharing similar features, but being ahead in terms

of planning and M&E. Also, relevant professionals of key departments were sent on this study tour: the

Monitoring and Evaluation Undersecretary, the National Development Plan Results Monitoring Director,

and the National Budget Monitoring Director.

The NECD intervention was effective and efficient to a significant extent since the participants in the study

tour learnt a lot of relevant information in a short time and at a low cost. Output targets were largely

achieved since they had fruitful discussions and liaisons with counterparts in the Planning Department of

Colombia (Sinergia) to learn how to identify priorities in terms of evaluation and elaborate an evaluation

agenda to propose to the National Planning Board of Ecuador. This short study tour also did not create

problems from a long absence from work.

This NECD initiative translated into immediate outcomes as Ecuador defined a national evaluation plan in

2015 with prioritized evaluations to be carried out that year. This evaluation plan was then validated by

the Planning Commission led by the President. Those immediate outcomes seem to translate into final

outcomes (to the extent the latter are yet to materialize given the short time span) as Ecuador continues

to develop an annual evaluation plan that allows to prioritize evaluations and allocate resources to


evaluations. One challenge though for sustainability is staff rotation as all members of the mission to

Colombia have since moved to occupy other positions in the public sector.


The NECD intervention with key reference documents is high in general, such as the UNICEF Evaluation

Policy, the UNICEF Strategic Plan, 2014-2017, and the GA resolution on National Evaluation Capacity

Development and UNEG Norms and Standards. But some documents such as the GA resolution on the

Agenda 2030 and the Global Evaluation Agenda are not well-known by the respondents.


and SDGs

The NECD intervention considered all cross-cutting dimensions when discussing evaluation priorities.


Study tours are not costly and not too demanding in terms of organization and coaching. Contacts and

exchange among civil servants facing similar challenges are easy to establish.

The country selected to host the study tour must be well prepared and be relevant to the level of

development of the visiting country. Colombia was the right choice of country, ahead of Ecuador in

planning and M&E, but not too much. The Sinergia team is well organized and works seriously. Finally, the

study tour benefited from the support of UNICEF Country Offices in both countries.


Elaborating an evaluation agenda requires to have a broad vision of evaluation to consider various types

of evaluation depending on the expressed needs. The Ecuador delegation could expand its awareness and

knowledge of a menu of possible evaluations.

It is important to keep the flow of exchanges after the study tour and even more if some staff leave their

current position. UNICEF plays a key role to keep the momentum and do follow-up.


UNICEF enjoys a comparative advantage in terms of 1) experience in NECD, 2) expertise in NECD, and 3)

teamwork. UNICEF occupies a unique position because it is well respected and can exercise leverage on

other actors to induce them into engaging into evaluation. In this specific case, UNICEF created the

conditions so that key actors in planning and M&E from Ecuador could learn from the experience of their

peers in a similar country, but with more experience. UNICEF brings in external points of views and helps

counterparts to “think out of the box”.



UNICEF should continue its support to NECD in Ecuador over the medium and long run because the

evaluation culture in the public sector is still new in this country. This support should go to a variety of

public institutions, but also other institutions such as universities, research organizations, consultants,

multilateral organizations. In particular UNICEF can play a unique role in creating or supporting events and

spaces related to evaluation where those actors can share their experiences and results and benefit from

those of others in the country and elsewhere in the region and the world

Final conclusion

This study tour of Ecuador high level civil servants to visit their counterparts in Colombia was a low cost

NECD intervention with concrete immediate outcomes. The Colombia case was relevant for Ecuador and

could translate into a direct transfer of technical and institutional capital as the sharing of their approach

and tools to select evaluation priorities and elaborate an evaluation agenda with the corresponding

budget led Ecuador to adopt a similar process but adapted to their political and institutional realities. This

was individual NECD that led to institutional and enabling environment NECD. While a small player in both

Ecuador and Colombia, UNICEF Ecuador demonstrated it could contribute to the institutionalization of

the evaluation function of Ecuador and keep the momentum going in a context of government cycles.


CASE STUDY OF NECD INTERVENTION: CAPACITY BUILDING AND INSTITUTIONAL CONDITIONS TO INTRODUCE AND M&E

SYSTEM IN KYRGYZSTAN

Note: this NECD intervention is a combination of two projects conducted sequentially: “Improvement of

capacity building to establish a Monitoring and Evaluation system in Kyrgyzstan” in 2014 and “Formation

of institutional conditions and capacity to introduce an M&E system in Kyrgyzstan» in 2015.


Responsible agency or unit National Monitoring and Evaluation Network of the Kyrgyz Republic

through a (legally registered) Institute of Human Projecting (NGO)

Location Bishkek, Kyrgyzstan

Beneficiaries

Members of the National M&E Network of the Kyrgyz Republic (Kyrgyz

VOPE), Office of the Government, National Parliament,

representatives of government bodies responsible for implementation

of M&E, professional evaluation organizations, Central Asian and

international organizations engaged in evaluation

Partners OSCE, UN Women

Main objective

Enhance national capacity to introduce and implement an M&E system

in the public sector




months)

36


This NECD intervention had overall good performance. Its relevance is definitely high since M&E was a

relatively new concept in Kyrgyzstan and there is greater demand for improved performance,

transparency and accountability of the public sector. The intervention was effective with multiple outputs

and high value for money to raise the level of awareness of three important actors: Government,

Parliament, and the VOPE. The organization of an International M&E conference in Bishkek provided the

foundation to raise the awareness of Government and Parliament on the importance of M&E. Then the

intervention supported the participation of one government official and two Kyrgyz VOPE members in the

biannual conference of the European Evaluation Society in Dublin which helped to forge a long-term

partnership and establish contacts that proved useful during later stages of collaboration. Several

publications were produced under the NECD intervention: two analytical papers on an organizational and

legal analysis of the national M&E legislation and on the implementation of the M&E function in several

ministries, and methodological guidelines for the introduction of staff evaluation in the government.

Several joint capacity building events of civil servants and members of the Kyrgyz VOPE were organized.


Then pilot evaluations of a government program and staff appraisal were conducted. And an M&E law

was adopted in 2014

Those outputs have led to several outcomes, in particular an increased awareness of key national

stakeholders of the importance of M&E in the public sector, a greater commitment of the Kyrgyz

Government and Parliament to the SDG Agenda, the establishment of solid partnerships with Parliament,

Government officials, and other national and regional stakeholders, and an increased reputation of the

Kyrgyz VOPE.

However, sustainability remains a challenge. At individual level, there is no systematic M&E training for

civil servants. At institutional level, the capacity of many ministries to actually implement and use the

M&E system is week and there is a need to involve academic institutions to offer the trainings. At enabling

environment level, the unstable political environment and frequent changes in the government can affect

the continuity of processes.


There is overall alignment of the NECD intervention with major reference documents such as the GA

resolution on National Evaluation Capacity Development, the GA resolution on the Agenda 2030, the

Global Evaluation Agenda and UNEG Norms and Standards and partial alignment with the UNICEF

Evaluation Policy and UNICEF Strategic Plan.


and SDGs

This NECD intervention addressed all those crosscutting dimensions.


The engagement of Government and Parliament and non-government actors like the VOPE proved to be

important for a long-term partnership going beyond short-term donor-supported interventions. The

leading role of national MPs who are also members of the Global Parliamentarians Forum for Evaluation

(GPFE) facilitated the organization of events and ensured a wide interest in both Parliament and

Government. This demonstrated a multiplier effect of combining the activities of the GPFE global initiative

and of the CO support.

Finally, the importance of building individual capacities of evaluators by high-quality international

instructors in areas where there is not enough national capacity in this area proved crucial for progress.



It has been highlighted that one single NECD intervention may not change the evaluation capacity

situation in the short and medium run, hence the need to expand the scope and sustainability of results

through a programme and a portfolio perspective.


One comparative advantage of UNICEF is related to its openness to collaboration and the inclusion of

NECD as a priority for the SDGs agenda. UNICEF has a strong and sustained relationship with both

Government and Parliament and enjoys a positive reputation in the country. UNICEF proved to be well-

organized, with a demonstrated capacity to lead, support by the HQ Evaluation Office, and backed up by

significant funding.


UNICEF should continue its support to NECD in Kyrgyzstan over the medium and long run because the

evaluation culture in the public sector is still new in this country. UNICEF has strong comparative

advantages in this domain, most notably through its longstanding presence in the country, strong

technical expertise and a close relationship with the key MPs which needs to be considered as a strategic

point for future NECD. As a matter of fact, awareness has been raised to a point where those MPs consider

the evaluation of programmes as a good approach to boost accountability of ministries and governmental

institutions. UNICEF support is needed to address issues of institutionalization and legitimation of the

M&E process, as well as the customization of tools, which still requires time, expertise, and financial

resources.

Final conclusion

The outputs of this NECD intervention have been deeply appreciated as expressed directly by the key

beneficiaries: UNICEF has been “the sole development agency that has been consistently providing support

to NECD activities over several years, which has been instrumental in the achievement of most of the

outcomes and impact and has shaped the NECD situation in the country”. Kirgizstan is a country where

continued actions for NECD at individual, institutional, and enabling environment levels are needed to

obtain sustainable outcomes. The opportunity cost of letting go after critical support to build the

foundations of an M&E system is deemed very high.


CASE STUDY OF NECD INTERVENTION: SUPPORT TO THE CULTURE AND INSTITUTIONALIZATION OF EVALUATION IN

MOROCCO


Responsible agency or unit Social Inclusion Section of the Moroccan Association of Evaluation

(AME)

Location Rabat

Beneficiaries

Members of the AME, ministerial departments, Parliamentarians,

elected decentralized actors

Partners Ministerial Departments, Parliament and NGOs

Main objectives

1. Strengthen the culture of evaluation

2. Strengthening national capacities for Policy Evaluation

3. Advocacy for the institutionalization of the evaluation

Actual start year January 2013

Actual or planned end year December 2017


months)

60 months


This NECD intervention demonstrates overall good performance. It is relevant since it promotes individual

capacity development of key actors. It has been effective and efficient in producing outputs, i.e. increased

program evaluation capacities of Parliamentarians, elected decentralized actors, and ministerial

departments through the organization of Moroccan Evaluation Week every two years, organized with a

focus on the evaluation of policies targeting children; and South-South cooperation activities to

strengthen the field of evaluation. The outcome is related to the sharing of good practices in evaluation

of social policies targeting children.


There is overall alignment of this NECD intervention with key reference documents such as UNICEF

Evaluation Policy, GA resolution on NECD, GA resolution on the Agenda 2030 and Global Evaluation

Agenda, the UNICEF Strategic Plan, 2014-2017, and UNEG Norms and Standards.


and SDGs

This NECD initiative took into account human rights, equity, and SDGs.



Good practices noted in this NECD intervention include: i) the organization of national evaluations events

to promote the continuous training and collaboration of stakeholders, ii) the collaboration of additional

actors such as the community of practitioners, Parliamentarians and decentralized actors; and iii) the

integration of SDG evaluation issues during the second francophone forum of evaluation held in Morocco

in 2016. These activities proved to be useful to raise awareness and build upon networking capacities.


One lesson learnt from this NECD intervention is related to the opportunities created through the process

of promoting the exchanges of knowledge, experiences, and improved practices, either South-South or

South-North. UNICEF needs to seize the opportunity to strengthen and reinforce local capacities to unify

the approach and language of evaluation policies.

Another lesson is the limit of working mainly in terms of NECD at individual level. The AME could achieve

broader results by working more at institutional and enabling environment levels. This would imply a

closer partnership with central level national partners such as the Ministry of General Affairs and

Governance (MAGG), the National Observatory of Human Development (ONDH) and Government General

Secretariat.


The comparative advantage of UNICEF lies in its focus on children, the support in terms of expert

mobilization and networking facilitation, and finally, the support to the advocacy effort for the

institutionalization of public policy evaluation in Morocco.


Recommendations for UNICEF are related to supporting the AME and other actors in Morocco to reinforce

their capacity to communicate about the main results of their work in evaluation to capture the attention

of other actors that might be interested in supporting NECD activities to enhance the networking and to

foster more collaboration and partnerships among Moroccan actors. It may require developing broader

partnerships, including with actors that have a better networking capability with key central government

institutions to enhance the results in advocacy for more institutionalization of evaluation. Further work

with sub-national actors would also enable to address the specific issues in each region of the country.

Finally, UNICEF could also advocate more for the consideration of equity and gender equality in its NECD

activities in a context of rising social issues.


Final conclusion

This NECD intervention has created a real space to bring together the stakeholders, the community of

practices and the government in a common effort to share policy evaluations and to reinforce the capacity

to communicate and work around a common approach in the country. Achieving final outcomes in a

sustainable way to enable a culture of evaluation to permeate the public sector would imply going one

step further than individual evaluation capacity development and working more at institutional and

enabling environment levels and fostering broader partnerships, in particular with key central

government institutions and intensifying work with sub-national actors.


CASE STUDY OF NECD INTERVENTION: NATIONAL EVALUATION CAPACITY DEVELOPMENT IN NIGERIA: A TWO-FOLD

STRATEGY

Note: this NECD intervention includes two components: Regulatory framework for the evaluation of

public sector programmes/projects; the support to the creation of the Nigerian Association of Evaluators

(formerly the Association of Nigeria Evaluators).


Responsible agency or unit Ministry of Budget and National Planning

Location Abuja and several other cities

Beneficiaries Ministries, civil servants, and services delivery users.

Partners

Ministry of Budget and National Planning, National Bureau of Statistics,

Special Advisors to the President on SDGs, Association of Nigerian

Evaluators UNFPA, ILO, WHO, World Bank, Nigerian Economic Summit

Group, representatives of private sector, academia and NGOs.

Main objectives

1. To strengthen the capacity of Ministry of Budget and National

Planning (MBNP) with the Evaluation Function, Federal MDAs and

State MDAs

2. To support for the establishment of the Nigeria Association of

Evaluators




months)

48


The performance of this NECD intervention is high. It is obviously relevant as it aims at raising the profile

of the evaluation function within the government, working on advocacy and developing a NEP (demand);

and creating a National Evaluation Association (demand and supply). Hence the title of “two-fold

strategy”.

It has been effective in the stepwise provision of outputs, building the basis for sustainable outcomes

through a participatory and country-owned process. UNICEF and the Ministry of Budget and National

Planning (MBNP) worked together to support the unification in a federation of associations known as the

Nigerian Association of Evaluators (NAE) that functions as the national VOPE in Nigeria. The NAE aims to

further develop and promote a community of evaluators and practitioners that will support the

institutionalization of evaluation in Nigeria. It was instrumental in advocating for a National Evaluation

Policy, and for government officials to engage with the professional communities of evaluators. To further

build the evaluation function, UNICEF and MBNP continue to support NEA to organize annual conferences


to raise the importance of evaluation as an important public management function. The NAE platform

was used to organize three conferences on Evaluation Function for Evidences based Policy Making, SDGs

Achievement, and Accountability) and publish three high level declarations: to adopt an Evaluation Policy

and Develop a Result based management framework in 2015, to continue the engagement toward such

goals in 2016, given the change in government, to maintain continuity with the new government agenda,

to obtain a National Agreement towards the Development of a National Accountability Framework to

support the Monitoring and Evaluation functions of government partners in achieving and reporting on

the SDGs In 2017.

Although it is too early to talk about final outcomes, those outputs have clearly already translated into

immediate outcomes with the commitment of the government to adopt the National Evaluation Policy

(NEP) and being more engaged with strengthening the evaluation function. The next steps are to i)

legislate this policy as an executive decree in the short run and as Law in the medium run; and ii) support

the MBNP with a Monitoring Policy.


The NECD intervention is aligned with major reference documents, including UNICEF Evaluation Policy,

GA resolution on NECD, GA resolution on the Agenda 2030 and Global Evaluation Agenda, the UNICEF

Strategic Plan, 2014-2017, UNEG Norms and Standards as well as the EVALPARTNERS Global Evaluation

Agenda.


and SDGs

The NECD intervention considered equity and SDGs as cross-cutting dimensions.


The main good practice was the unification of the national evaluation leaders under a national VOPE that

serves as an umbrella organization for evaluation associations in Nigeria, demonstrating the benefits of

coming together in a context of a high, but scattered supply of evaluators, with limited or no opportunity

for networking. The NEA, with the support of key actors like UNICEF, will be in a position to further develop

and promote a community of evaluators and practitioners that will support the institutionalization of

evaluation through capacity building activities.

Let us also mention key factors for sustainability which is sometimes a weak point of NECD interventions:

the focus to sustain an enabling environment, advocacy for more country-led evaluations and further

professionalization of the evaluation function, continued engagement and support to the MBNP, regular

engagements with the regional evaluation association in Africa (AFREA) and with global evaluation

associations (IDEAS), as well as peer review mechanisms to rollout the NEP.


Finally, let us mention the importance of documenting the experience of the NECD intervention which has

been the case in Nigeria32.


NECD takes time to be sustainable and country-owned and the kind of stepwise and participatory

approach used by UNICEF has proven fruitful. The endorsement of NEP should be considered as a key

milestone for the enabling environment, but this is not necessarily the only way forward for large

countries like Nigeria with significant pockets of competencies not only at national level, but also at sub-

national level (States). There is a big potential due to the availability of technical organizations working in

the evaluation area and UNICEF has been good in tapping them and helping bring them together. Also,

the strategic partnership and quality of the relationship with the MBNP is very valuable for NECD

interventions.


UNICEF is appreciated playing a convener role, bringing development partners to work together, and

contributing its global leadership experience, support to networks such as EVALPARTNERS and its support

to conferences and events promoting South-South cooperation. Another advantage is the good rapport

of the CO with the MBNP which facilitated a number of initiatives while keeping a neutral position vis-a-

vis the government and government partners, and its support to government systems.


The obvious recommendation for UNICEF in Nigeria is to continue its crucial support to combine different

aspects under the NECD intervention and merge the efforts of the government, the NAE, and other

stakeholders, contribute to the dissemination of evaluation methodologies and good practices at federal

and state level, and build up the technical capacity of staff in key Ministries, Departments and Agencies

(MDAs) using a Training of Trainers approach, including a hands-on approach of conducting Joint

Evaluation studies with the government and the NAE.

The recommendation to other country offices is to consider using the same two-fold strategy to create

the conditions for more institutionalization of the evaluation function. The recommendation to the

regional offices is to continue and increase experience sharing in the region, promoting exchange

programs among countries that have similar contexts.

Finally, UNICEF could also advocate more for the consideration of human rights and gender equality in its

NECD activities.

32 Jobin, D. and Z. Lawal (2017). “Application of game theory and new institutional economics in establishing a

National Voluntary Organisation for Professional Evaluation in Nigeria”. African Evaluation Journal. https://aejonline.org/index.php/aej/article/view/197


Final conclusion

Nigeria provides an interesting case study of a large country with significant human and financial

resources, but relatively new to a culture of transparency, evaluation, and accountability as well as facing

issues of coordination at national level, between the federal government and the state governments, and

between government and non-government actors. UNICEF has supported a two-fold strategy working the

national government and the evaluators to create a national VOPE which has been instrumental in

promoting the importance of the evaluation function and advocating for a National Evaluation Policy as

well as to engage government officials with the professional communities of evaluators. Major

conferences have been organized on a regular basis, leading to high level declarations. NECD is a long-

term process, but building an enabling environment is an essential ingredient.


CASE STUDY OF NECD INTERVENTION: NATIONAL EVALUATION POLICY IN THE PHILIPPINES


Responsible agency or unit National Economic and Development Authority (NEDA) and

Department of Budget and Management (DBM)

Location Manila, Philippines

Beneficiaries Public sector

Partners

Main objective Crafting of National Evaluation Policy of the Philippines




months)

36


The performance of this NECD intervention is high in terms of relevance, effectiveness, efficiency and

outputs. The process of elaboration of the draft National Evaluation Policy took some time which is not

unusual given the importance of this orientation document, the number of stakeholders, and the

resistance to change of some stakeholders. The Policy issuance of the National Evaluation Framework of

the Philippines was done in July 2015 in a joint memorandum NEDA-DBM. In addition to this consistency

framework stating clearly the importance of evaluation and the roles of key stakeholders, the

participation of many public sector actors in fora, public hearings, and support task forces also contributed

to improve the enabling environment for the purposive conduct of evaluations in the public sector in

support of good governance, transparency, accountability, and evidence-based decision-making. Along

with the Monitoring, evaluation and reporting policy put forward by DBM, also benefiting from the

support of UNICEF, to provide clear guidelines at operational level between DBM and line ministries,

awareness among government agencies has increased to a significant extent and many agencies have

started instituting evaluation plans.

Although it is too early to analyze final outcomes, there are some concerns about sustainability. First, the

change of administration in 2016 meant a period of flux in policy making with the review of the previous

administration policies. Also, the actual implementation of the policy implies building up the capacities of

line ministries in managing and/or conducting evaluations.



The NECD intervention had a partial but direct alignment with all relevant documents listed. It is an

intervention at enabling environment level which facilitates the connection with global strategic

documents and policy guidelines.


and SDGs

The NECD intervention took into account equity, gender equality, and SDGs as crosscutting dimensions.

Finally, UNICEF could also advocate more for the consideration of human rights in its NECD activities.


This NECD intervention illustrated ways to minimize the impact of policy cycles and unexpected events.

Good practices included advocating for the government buy-in and engagement at early stages, and

promoting ownership by involving the government agencies at every step of the process. This also

involved the identification of champions within the partner government agencies; and a non-political,

non-partisan engagement when conducting NECD activities. (watch out: the previous sentence said the

opposite of what you wanted to say)


The overall NECD agenda should consider political changes and policy cycles which affect directly the

availability of internal “champions” for full policy implementation. This speaks of the importance to

involve other UN agencies in the country (like a ONE-UN/UNDAF agenda) and build strong strategic

partnerships to improve support. UNICEF managed to promote the objectives of the NECD intervention

apart from political objectives.

The articulation of the NECD intervention with the national strategies and policies such as the National

Development Plan also helps the buy-in of national stakeholders towards the intervention.

NECD is to be considered in the medium run. The implementation of the policy requires further support

and capacity-building of actors, including the community of M&E professionals in Philippines.


The main comparative advantage of UNICEF is its ability to put together a well-established network which

includes government, academia, other non-public sectors actors, and other development agencies.

UNICEF has a partners-built system for NECD which allows national actors to benefit from this know-how

and accumulated knowledge with capacity development material like toolkits, IEC, guidelines, etc.



UNICEF should continue its support for the implementation of the NEP. This implies building individual

and institutional capacities in many agencies and other actors at national and sub-national level, which is

a huge task. The first proposal is to coordinate with other actors involved in NECD interventions, in

particular UN agencies, for pooling together resources and designing more ambitious complementary

interventions for bigger results. The second proposal is to use pilot agencies, programmes, provinces and

municipalities with progressive scaling up. Finally, UNICEF could share good practices from the region and

from other regions in terms of implementation of NEPs.

Final conclusion

This NECD intervention is a good example showing that working in good partnership with the government

central planning and budgeting agencies such as NEDA and DBM proved essential to push the elaboration

of a government wide policy on evaluation. It also demonstrated how to overcome policy cycles and

unexpected events which can be a major threat for the sustainability of NECD. UNICEF’s approach to

finding and nurturing linkages with the new power holders, its close partnership with a variety of actors

(public agencies, academia, practitioners, evaluation associations, parliamentarians, etc.) in a non-

political approach, its support to make them work together for the NECD in a coordinated way were key

to overcome political cycles and obtain results.


CASE STUDY OF NECD INTERVENTION: ESTABLISHMENT OF THE RWANDA MONITORING AND EVALUATION SOCIETY


Responsible agency or unit UNICEF Rwanda PME section/ Capacity Development and Employment

Services Board (CESB)

Location Rwanda, Kigali

Beneficiaries

Rwanda M&E Society (RMES) and all persons with professional practice,

knowledge and interest in M&E who subscribe to the objectives of the

RMES

Partners

Rwanda National Capacity Building Secretariat, National Institute of

Statistics of Rwanda (NISR), Ministry of Labour and Ministry of Finance

Main objectives

1) To support an institutional set up of Rwanda M&E Society

2) To establish a functional Rwandan monitoring and evaluation

society (RMES)

3) To cultivate the highest professional standards in Monitoring and

Evaluation practice and theory through highly skilled and

motivated members


Actual or planned end year Still running


months)

Ongoing


This NECD intervention is still in an initial stage and performance cannot yet be fully assessed. However,

it is interesting to document how it emerged and the role of UNICEF. It is obviously relevant to establish

a VOPE and UNICEF has proven quite effective in sizing the opportunity of a heightened awareness

environment at this time to help deliver outputs. Early 2016 UNICEF Rwanda PME section started

advocating for national evaluation and RBM capacity building with the government and the community

of donors, which led to a discussion on the establishment of the country’s first M&E Society with the

Belgium Embassy and the National Capacity Building Secretariat (NCBS). In November 2016, UNICEF, NISR,

the Ministry of Labour and the Ministry of Finance met and agreed on the establishment of the country’s

first M&E society. Since then, UNICEF has been closely supporting NCBS, a custodian of this initiative. As

of the end of 2017, the institutional framework of the Rwanda M&E Society (RMES), i.e. the Statutes and

draft Annual Workplan 2018 had been drafted. The executive committee members were democratically

elected. The capacity of the Executive Committee has been enhanced through the support of the African

Development Bank and the organization of three high level training workshops in Rapid Results Approach

and Managing for Development Results. Office set-up for the M&E Society Secretariat is supposed to take

place in 2018


This NECD has so far been efficient, UNICEF spending little financial resources during the initial setup.

Resource needs are likely to increase with the operational support to be provided to ensure a functioning

RMES and, if not addressed, might be a barrier for the sustainability of the increased evaluation capacity,

participation of M&E practitioners in the overall country development, and increased interaction and

information sharing among all those interested in M&E. However, the engagement with the national

government institutions, civil society, and academics is a reassuring factor.





Evaluation Policy and UNICEF Strategic Plan. This is perfectly normal since by definition the NECD

intervention concerns only a limited set of activities while the documents cover a large scope of action.


and SDGs

The NECD intervention considers human rights, equity, and SDGs as cross-cutting dimensions.


The obvious good practices are the national ownership of the RMES where 100% of the Committee

members are Rwandan professionals, inclusive of all major government actors, and its participatory and

democratic process in society management. The election of national professionals as executive committee

members of elected provides an anchoring in local know how. Also, once obtained, the strong leadership

of the government provides a major impetus to the organization of NECD activities. Finally, an interesting

practice proposed (not yet implemented) is to track behavioral and institutional changes following NECD

activities through an online survey.


Advocacy for evaluation can be achieved with limited resources, but the sustainability of activities,

outputs, and outcomes requires a mid-term investment. Technical support and mentoring are important

in order to harness the national ownership.


UNICEF comparative advantage in NECD comes first from its capacity to engage different types of

partners, i.e. government, and non-government - NGOs, academics, private sector, etc., and to provide a

platform for interaction and information-sharing among all those interested in M&E. Second, UNICEF is a


knowledge-based organization which can easily transfer skills to the locals with high quality intellectual,

ethical and professional standards in M&E.


The situation in Rwanda would benefit from a leading support role of UNICEF working through the RMES

and the Rwanda MfDR Community of Practices initiated in October 2017. The timing is adequate to target

actions in line with the implementation the 2030 SDGs global agenda.

One domain of recommendation to UNICEF is related to the enhancement of communications and

documentation of the NECD activities. It is important to keep documenting on a regular basis good

practices and communicate to stakeholders the results arising from the NECD interventions. This is to be

done by the CO, but also by the RO which can facilitate experience exchange among UNICEF COs which

are supporting NECD initiatives for cross-fertilization.

From an operational point of view, the recommendations for the UNICEF CO are to: 1) help set up an NECD

seed fund to increase the scope and sustainability of NECD interventions, including research; 2) choose

carefully strategic NECD activities to be supported, considering building capacities at individual,

institutional, and enabling environment levels; 3) explore other partnerships to coordinate and liaise with

the Capacity Building and Employment Services Board (CESB) to promote a culture of results in Rwanda;

and 4) take advantage of the RMES to engage the government in developing a National Evaluation

Framework and if possible a National Evaluation Secretariat.

Final conclusion

Rwanda is at the initial stage of NECD and it is too early to assess the full performance of NECD

interventions. However, UNICEF has helped this country start on the right foot with strong national

empowerment and leadership, and involvement of key government and non-government actors. The

creation of the RMES has generated a momentum to engage into strategic M&E activities beyond just

individual training, in particular the establishment of a National Evaluation framework or policy. Also,

Rwanda as a latecomer to NECD can learn from the other regional and international experiences from

more advanced countries in NECD, avoid mistakes, grab good practices, and catch up quickly, if not double

those other countries.


CASE STUDY OF NECD INTERVENTION: STRENGTHENING NATIONAL M&E SYSTEM IN SRI LANKA


Responsible agency or unit UNICEF CO

Location Sri Lanka – national and subnational (district) levels

Beneficiaries Parliamentarians and public officials

Partners

Global Parliamentarians Forum for Evaluation (GPFE, ex PFDE) at global

level and Parliamentarians’ Forum for Evaluation in South Asia and in

Sri Lanka; Organization of Environment and Children Rights

Preservation (OECRP); Department of Project Management and

Monitoring at the Ministry of Development Assignments; Sri Lanka

Evaluation Association; EVALPARTNERS; University of Sri

Jaywardenepura

Main objective

Strengthen the enabling environment for evaluation and build the

evaluation capacity to promote accountability and evidence-based

policy making and facilitate the achievement of SDGs in Sri Lanka


Actual or planned end year Ongoing


months)

18

Performance of the UNICEF intervention

The performance of this NECD intervention has been good overall. It is directly relevant to support the

strengthening the enabling environment and build capacity as the NECD intervention outputs were

targeted properly to reduce the gaps. Despite the fact that Sri Lanka is considered as a well-developed

M&E system, the practice of evaluation has not yet been fully institutionalized. The outputs of this NECD

were strategic and supportive of this institutionalization: i) a formulated National Evaluation Police (since

2003) and equity-focused and gender-responsive Evaluation Guidelines/ Standards; ii) Sri Lanka

Parliamentarians’ Forum for Evaluation established and operational to support active Members of

Parliament’ (MPs) demand; iii) District Coordinating Committees in 12 targeted districts sensitized in SDG

and use of evaluation at subnational level; and, iv) capacity building, in partnership with the University of

Sri Jayawardenepura, with a post-graduate diploma programme in Evaluation launched and fully

operational.

Overall, Sri Lanka is a good example of country-led, systemic national evaluation capacity development.

And the CO has managed to keep the momentum going through strong partnerships (MPs, public officials,

independent evaluators and academia) and benefiting from a strong drive in the Parliament. However, at

outcome level, there are challenges that halt the results and not necessarily are under the range of action

of UNICEF, for example the NEP is still pending for approval at Parliament and institutionalization of


evaluation capacity was affected by changes in ministerial setup. Even if the NECD intervention has

increased individual and institutional capacities, those capacities are not necessarily put into practice at

enabling environment level. This creates challenges for sustainability at this level.

Extent to which current the NECD intervention is in line with important reference documents




Evaluation Policy and UNICEF Strategic Plan.


and SDGs

This NECD intervention addresses all crosscutting dimensions. Let us mention specifically the elaboration

of evaluation guidelines on gender responsive and equity focused evaluations and capacity building of

target groups in the use of those guidelines.


One good practice has been the interconnected approach of the CO. It prepared a good needs assessment

and has directed strategically activities to target the gaps, taking advantage of the level of development

of M&E in the country context. It also targeted strategic partners, government and non-government, in

evaluation; and built long term relationships. Another good practice has been to involve the subnational

level, where potentially better SDG-related services coverage is required. Also, the consultative process

has contributed to dissemination and raised awareness of the importance of NECD.

Finally, let us mention the partnership with the University of Sri Jayawardenepura to develop a post-

graduate diploma programme in Evaluation and the development of a set of user-friendly resource

materials, including guidance on equity-focused/gender-responsive evaluations, resource materials for

MPs on importance and use of evaluation supported by awareness and capacity development sessions

for MPS and the Parliament’s Research Unit.


A first lesson is the importance of strategic and well-built partnerships at all levels as well as multiple entry

points to be able to keep the momentum created by the portfolio of NECD interventions and pass through

more difficult periods linked to political cycles. A second lesson is the need for a good ex ante assessment

of the potential risks and identification of mitigation measures before deciding to go for an NECD

intervention.



UNICEF Sri Lanka CO comparative advantage is utilizing strong partnerships at global, regional, national

and subnational levels. It used a strategy of strengthening its partnerships with key government and non-

government institutions involved in M&E in the country like: Department of Project Management and

Monitoring (DPMM), Global and Sri Lankan Parliamentarians’ Fora for Evaluation, Department of Census

and Statistics (DCS), Sri Lanka Evaluation Association (SLEVA), UNDP as well as international networks

(EVALPARTNERS, EVALSDGs, etc.). At the same time, new partnerships have been developed – with one

of the leading universities in Sri Lanka and the Government ICT agency mandated for implementation of

all the ICT projects in the country.

The close alliance with the provincial and national governments and partners allow the organization of

capacities and joint efforts, even in financially scarce scenarios.


Institutionalizing successes in NECD would be facilitated by an explicit statement in the UNICEF Evaluation

policy and then corresponding resources allocations for ROs and COs. NECD cannot depend only on

fundraising by CO, otherwise the scope of the interventions will be constrained and the momentum might

be reduced, if not lost. UNICEF would also gain in having a clear vision on: i) what are the most relevant

NECD approaches; and ii) how they can be implemented with available resources together by HQ, RO, CO

as a team. Dissemination of information and experience within UNICEF is also important. The

recommendation is to provide some space and resources for documenting and sharing successful NECD

experiences, good practices, lessons learnt. Finally, the identification of criteria to decide for a “go or not

go “would help be more systematic in the assessment of opportunities, partners, and risks.

Final conclusion

Overall, this NECD intervention has been successful due to the fact that UNICEF and key partners have

used the momentum to permeate the demand of key actors like MP and Academia. Also, they promoted

to put into practice the developed guidelines by the target audience at parliament. Even though the NEP

is still pending for approval, the CO keeps on track with a vision to interconnect efforts. In this roadmap,

In September 2018, Colombo will host the Parliamentarians’ Forum on Evaluation with parliamentarians

from all over world, members of the evaluation community, government champions and development

partners. The office has been documenting the last two years of engagement in strengthening evaluation

in Sri Lanka, also to be used to support Sri Lanka’s National Voluntary Review submission at the High Level

Political Forum in July 2018 as part of the SDGs Follow-up and Review (FUR) Process. Sustainability remains

a challenge to overcome the instability and turnover created by political cycles, but the variety of

government and non-government partners with good long-term relationships, and a better analysis of

success factors for each NECD intervention could help address this challenge.


ANNEX 6 – EXAMPLE OF POSSIBLE GO/NO GO GRID AT COUNTRY LEVEL

The purpose of this grid is for pre-feasibility assessment. For a full ex-ante evaluation, consultants suggest a process like IDEA-Solutions Projects

for Results (P4R®). Among the criteria to be considered are the classical OECD/DAC criteria of relevance, anticipated effectiveness, efficiency,

results, and sustainability, but also the existence of a logic model based on a clear Theory of Change, the specificities of the country (i.e. NECD in

middle-income country vs low income country), the analysis of potential partners, the contribution of the NECD intervention in the NECD process,

the child-focus. Most of the NECD budget should be spent on those selected interventions, but with continued and significant institutional,

technical, and financial support guaranteed over time to get sizable results over the medium run.

NECD INTERVENTION GO/NO GO GRID AT COUNTRY LEVEL

Name of Potential UNICEF Supported NECD Intervention:

Name of the person, position, email filling the grid:

Date of the update:

1. Presentation of the potential NECD intervention

Target group:

Main Objective:

Main Components/Activities:

Anticipated Budget:

Anticipated Outputs:

Anticipated Outcomes:

Anticipated Duration:

Partners (if any):

Location:


2. Assessment grid

Criteria Weight* Evaluation

Justification -2 -1 0 1 2

A. Level of development of evaluation capacities from the countries expected to be part of the NECD 25%

10

NECD alignment with the Country's strategic orientations/strategic plan/etc.

Counterparts are clearly stablished

Favourability of the political cycle

Maturity of legal framework (norms/laws/constitutions)

Maturity of institutional framework (Clear M&E roles, M&E processes, M&E systems)

Subtotal

0 0 0 0 0

% points for A 0

0



B. Expressed commitment from Target Group to achieve outputs and outcomes

25% 10

Buy in from head officials/head positions

Buy in from technical officials/intermediate positions

Background and/or previous work with the partner (e.g. previous work in NECD with UNICEF)

Clarity upon expressed and documented outcomes

Willingness to endorse a performance agreement document

Subtotal

0 0 0 0 0

% points for B 0.0%

0

*Note: the weights may vary depending on the type of country (e.g. lower, middle, upper income)




C. Clarity in the contribution of NECD 25% 10

The contribution of UNICEF to the NECD intervention is clear

This contribution is feasible from a Human Resources point of view

This contribution is feasible from a material and financial resources point of view

This contribution is consistent with UNICEF mandate and country management plan

Subtotal

0 0 0 0 0

% points for C 0

0



D. Resources capacities of the target group 25% 4

The partner has the specific expertise (individual CVs and Organizational CV, i.e. revise project sheets or portfolio)

Level of effort and budget allocated are considered sufficient to deliver the expected outputs (e.g. revised according a timeline and preliminary budget)

Subtotal

0 0 0 0 0

% points for D 0

0



E. Other partners' support 25%

Availability of financial and human resources to complement the support

Commitment to the contribution of outputs and outcomes

Subtotal

0 0 0 0 0

% points for E 0 0

Final decision NO GO GO