Independent Evaluation Office (IEO) Engagement with International Program for Development Evaluation Training (IPDET)

Independent Evaluation Office Networking and Participation Report

2

TABLE OF CONTENTS

I. Background .............................................................................................................................. 3

II. IEO participation at IPDET 2014 .............................................................................................. 4

III. On the margin meetings .......................................................................................................... 5

IV. Annex ....................................................................................................................................... 7

3

I. BACKGROUND

The International Program for Development Evaluation Training (IPDET) is a leading executive

training program that was established in 2001. As an initiative of the Independent Evaluation

Group of the World Bank and the Faculty of Public Affairs at Carleton University, an IPDET

flagship training program is offered each summer in Ottawa, Canada. The program is directed

at managers and practitioners and aims to provide the tools required to evaluate development

policies, programs, and projects at local, national, regional, and global levels. To date, the

program has graduated close to 3,500 participants from over 125 countries. IPDET is taught by

renowned international faculty and is co-directed by Ray Rist and Linda Morra Imas, both

leading evaluation experts and former Evaluation Capacity Building Advisors at the Independent

Evaluation Group of the World Bank.

Representing the Independent Evaluation Office (IEO) of UNDP and United Nations Evaluation

Group (UNEG), the Director of the IEO, who is also a Vice-Chair of UNEG, remains engaged with

IPDET as visiting faculty. He previously developed and co-delivered courses from 2008 on the

evaluation of governance with the Director of the IEO of the Global Environmental facility (GEF).

Engagement with IPDET remains important and useful, and the IEO will continue to build this

relationship.

Since joining UNDP in 2012 he has been presenting the lunch time guest lecture and

participating in panels and workshops sharing insights into the evaluation function and IEO of

the UNDP. For example, in 2013, the Director’s participation included a co-presentation at

workshops led by Dr. Ray Rist (on setting up M&E systems) and Professor RM Levin (on

evaluating governance in Africa). These interactive workshops included an exploration of the

use of relevant techniques, instruments and methodologies.

Many leading global and national actors were represented at IPDET 2014, including participants

from government ministries, national commissions, non-governmental organizations, private

sector, think tanks and international development agencies. For example, participants over the

fourth week (when the Director of the IEO was present) included senior representatives from:

Ministry of Environmental Protection China, Ministry of Health Ghana, Ministry of Planning &

Administrative Development Palestine, Office of the Public Service Commission South Africa,

International Atomic Energy Agency Austria, Gross Happiness Commission Bhutan, African

Union Commission Ethiopia, World Health Organization, Swiss Agency for Development and

Cooperation Mongolia, World Bank Group, The Danish Evaluation Institute, SOS Children’s

Villages Georgia, Caribbean Development Bank, Asian Development Bank, Aga Khan Foundation

Tajikistan, and International Development and Research Centre Canada.

4

II. IEO PARTICIPATION AT IPDET 2014

Key note address by Director

At this year’s summer training program, the Director of the IEO delivered a lecture entitled

“Independence, Credibility and Use’. The presentation shared experiences of the IEO and

UNEG in evaluation, emphasizing the importance of ensuring independence, credibility, and

utility in evaluation. For example, topics discussed included the significance of quality

assurance, the role of institutional linkage and capacity, and the need for stakeholder

engagement. Some of the key messages conveyed in the presentation included the notion that

nothing is too complex to be measured and that evaluations must address issues surrounding

efficiency and management performance. The declaration of EvalPartners on 2015 as the Year

of Evaluation was also introduced to participants. Powerpoint slides from the presentation are

included here:

Ipdet 2014 Indran

Naidoo.pptx

Overall, the lecture was well received by participants. The audience was engaged and attentive

to hearing about the work and experiences of the UNDP IEO and UNEG. Following the lecture,

members of the audience commented on the professional nature of the presentation and

relevance of insights shared to their own work. Questions posed related to questions of

independence and management uptake of recommendations. The presentation was video

recorded and will soon be on the international IPDET and other websites.

Participation of Director on a Heads of Agencies panel

As in previous years, the Director of the IEO participated on an international panel on

“Managing and Evaluation Unit” alongside the following:

Rob van den Berg (Director: IEO – GEF)

Goberdhan Singh (Former Director: CIDA)

Maliha Khan (Oxfam USA)

Niels Dabelstein (former Director, DANIDA Evaluation unit – and former Peer Review of

the UNDP Evaluation Policy)

The workshop addressed issues surrounding setting up a new evaluation unit, including issues

surrounding structuring, staffing, budgeting, and creating good products. Panel members were

5

asked to share knowledge and insights into these and other key issues. Discussion included

topics of structural and behavioural independence and management uptake of

recommendations.

Participation of Deputy Director and Staff Member of IEO, UNDP

In IPDET 2014, Juha Uitto, Deputy Director of the IEO, UNDP was a co-instructor with Robert

van den Berg (Director of GEF IEO) for a workshop on “Challenges in Evaluating Environment

and Development Issues”. The workshop tackled difficulties of scope, timing, and linkages

between climate change, natural resources management, poverty and development in

evaluations. The presentation covered some of the topics discussed in a recent book on

“Evaluating Environment in International Development”, edited by Juha Uitto with a forward by

Indran Naidoo

Roberto La Rovere from the IEO UNDP was a participant at IPDET 2014. Several staff from the

UN system (for example, UNICEF, UNEP, UN Women) and members of UNDP Country Offices

(for example Bosnia and Herzegovina, Congo and Zimbabwe) also participated in this year’s

program.

III. ON THE MARGIN MEETINGS

Planning for the 2015 National Evaluation Capacity (NEC) conference

Preparatory work is underway for the next NEC event that will be held in the Asia region in

September/October of 2015. The previous NEC event partnered with the Government and

Brazil and EvalPartners, and a proceedings has just been released.

It has been decided to extend the reach further for the next event, and plans are afoot to

partner with the International Development Evaluation Association (IDEAS). The IEO has

supported IDEAS previously, and participated in the IDEAS Global Assembly in Barbados in May

2013, where the Director presented a key note address on ‘Evaluation and its role in advancing

a measurement discourse’. Note: an article based on this address, ‘Evaluators as agents in

development change’ was published in the September 2013 European Evaluation Society

Newsletter (page five). The newly elected President of IDEAS, Rob van den Berg is keen to

participate on this, with a back-to-back event in the region, that shall make it the largest NEC

and also largest evaluation event in the UN, as it will bring together the global network of

development evaluation thinks on an event that will have training and other relevant side

events. This will be strategic from a funding and profile point of view.

6

Meeting with Professor Levin – Public Service Commission (PSC) Head, South Africa - to

discuss collaboration around evaluation training in Africa

As a follow-up to a meeting held in December 2013 at the PSC, the Director met with Professor

Levin to discuss a course proposal for an Africa initiative, linked to AAPSCOMS for a UN and

other training programme. Discussions were also held with Dr. Rist, who is keen to partner with

IPDET for this event. A proposal has been developed and has been shared with IPDET and the

Swiss, for possible seed funding for this collaboration.

Meeting with Dr. Rist on mini-IPDET for UN

A meeting was held to discuss the possibilities for developing a UNDP course for the regional

M&E officers. The IEO is also looking to develop its own training course in 2015, as it would

provide an opportunity to enhance the UNDP. There is a willingness to support this initiative,

and a proposal has been submitted for funding.

Meeting with Swiss on funding proposal

A meeting was held with Lucien Aegerter, the Programme Manager, Management for

Development results, of the Swiss Agency for Development and Cooperation (SDC). The IEO

submitted a proposal for funding a training course, and this was accepted in principle. The

purpose of the meeting was to discuss details and chart a way forward, on both direct support

to IEO, the UNDP and the regions.

Meeting with Dr. Rist as member of the Advisory Panel of IEO, UNDP

The opportunity was used to discuss IEO developments, and in particular the 2014-2017 work

plan and how members will support it. Dr. Rist will also contribute to the office on issues

relating to strategic positioning of IEO and UNDP.

Meeting with Ms. Jankovic on UNEG

A series of meetings were held between the Director of the IEO, in his role as the Vice-Chair of

UNEG, Evaluation function, and Ms. Judita Jankovic (International Civil Aviation Organization –

ICAO, Canada) who is the convener of the area. The UNEG work plan for this area calls for

deliverables on the area of professionalization, norms and standards and Peer Reviews. The aim

is to move towards certification of evaluators, and an occupational class within the UN, as is the

practice globally. The participation of Ms. Jankovic was important as she gained an

understanding of what needed to be done, and why. The other area of work – Peer Review is

proceeding well, and the Director of the IEO has just led a review of the evaluation function of

UNEG and a report shall be released shortly.

7

IV. ANNEX

Networking with participants after Director of IEO’s

lecture

Participants at the ‘Managing an Evaluation Unit’

panel.

Richard Levin, Indran Naidoo, Julia Tompson, and

Judita Jankovic

Linda Morras , Ray Rist and Richard Levin

8

Presentation by Indran Naidoo (Director, IEO)

Presentation by Michael Patton (Faculty, IPDET) Director of the IEO featured as ‘Speakers’ Lunch

Independent Evaluation Office

EVA

LUA

TIO

N

THE INDEPENDENT EVALUATION OFFICE OF UNDP

INDEPENDENCE, CREDIBILITY AND USE

IPDET, 30 JUNE 2014

Indran A. NaidooDirector

Independent Evaluation Office

Outline

Make evaluation count!

Dynamic triad of independence, credibility & utility

Experience from the IEO of UNDP

2

Independence

CredibilityUtility

Independent Evaluation Office

Evaluand

3

Country Presence

UNDP in 177 countries

Budget

Annual Budget is more than 5 Billion dollars

FOCUS

AREAS

Poverty alleviation; democratic governance; energy & environment; and crisis prevention & recovery

Independent Evaluation Office

Influencing the evaluand at all levels

IEO evaluations have influenced the new Strategic Plan of UNDP – 2014-2016 – thus all of UNDP work

90% uptake of IEO recommendations Management reflects in its documents how it has

addressed recommendations & updates public website

All new programmes must be preceded by and include response to an evaluation

4

Independent Evaluation Office

Recent Evaluations from IEO

5

Independent Evaluation Office

Recent Evaluations from IEO

6

Visit IEO website @ www.undp.org/evaluation

Independent Evaluation Office

The critical “how question”

Professional evaluation, what does it mean in practice

Principles, approaches and ensuring that the end is in sight

7

Independent Evaluation Office

Importance of the ‘Evaluation Triad’

Utility

Credibility

Independence

8

Independent Evaluation Office

Independent Evaluation Office in UNDP

PROCEDURESPOLICY

IEO work is guided by the Board approved

Evaluation policy

Evaluation Policy periodically reviewed

Management engagement

from inception to conclusion

Structurally Independent

Reports directly to UNDP Executive Boardwho appoints/renews/

dismisses Director

STRUCTURE

9

Independent Evaluation Office

Independence

10

Quality Assurance Policy review

Evaluation Advisory Panel

Peer Reviews

Reporting Line

Director reports to Executive Board to

prevent censorship and enhance transparency

INTERNALLY @ IEO Staff protected from

occupational detriment Director takes full

accountability

Independent Evaluation Office

Credibility

11

Credibility

Methodology

Guidance

Institutional Linkage and

Capacity

Professional Forums & Networks

Independent Evaluation Office

Utility

12

Uptake of recommendations• Management responses to evaluations and key actions publicly available in Evaluation Resource

Centre in IEO website• UNDP strategies and policies are informed by evaluations

(UNDP strategic plan 2014-2017)

Stakeholder engagement• Management engagement throughout the evaluation process• Stakeholder workshop organized for all evaluations to share its findings and recommendations.

Government, civil societies and all other stakeholders participate at these workshops

Organizational accountability & learning• Implementation of evaluation recommendation and associated key actions reported to UNDP

Executive Board in Annual Report on Evaluation• Assessment of Development Results(ADR) provides substantive in put for development of new

UNDP Country Programmes• All new country offices are required to submit fully costed evaluation plan with their new country

programme documents (CPDs)

Independent Evaluation Office

UN Evaluation Group (UNEG)

13

The United Nations Evaluation Group (UNEG) is a unique interagency professional network in the UN system that aims to advance the effectiveness, efficiency, impact and sustainability of the UN system’s work by promoting and strengthening evaluation. UNEG brings together units responsible for evaluation in the UN system, including UN departments, specialized agencies, funds and programmes, and affiliated organizations. It currently has 45 such members and three observers.

UNEG’s mission is to: promote the independence, credibility and usefulness of the evaluation function and evaluation across the UN system; advocate for the importance of evaluation for learning, decision-making and accountability; support the evaluation community in the UN system and beyond.

Independent Evaluation Office

Professionalization

14

Evaluation is emerging as a mature profession.

For evaluation to be taken seriously it needs to distinguish itself as a profession which requires policies, strategies and practices that bring changes at the agency and the staff level.

UNEG advances the professionalization of evaluation within the UN system and promotes adherence to the norms and standards through the external review processes of evaluation functions, the development of relevant guidance materials, as well as the development of a professional competency framework for UN evaluators.

Independent Evaluation Office

UNEG Strategic Objectives 2014-2019

15

(UNEG Strategy 2014-2019 available at: www.unevaluation.org/uneg_strategy)

Strategic Objectives

1

Evaluation functions and products of UN

entities meet the UNEG Norms and Standards

for evaluation;

Strategic Objectives

2

UN entities and partners use evaluation

in support of accountability and

programme learning;

Strategic Objectives

3

Evaluation informs UN system-wide initiatives

and emerging demands;

Strategic Objectives

4

UNEG benefits from and contributes to an

enhanced global evaluation profession.

Independent Evaluation Office

Engaging with governments

The National Evaluation Capacity development initiatives …

16

Independent Evaluation Office

National Evaluation Capacity

17

Independent Evaluation Office

National Evaluation Capacity

18

Independence

Credibility

Utility

- Rebeca Grynspan, Associate Administrator UNDP (2010-2014)

‘National evaluation capacities are increasingly critical to countries’ overall

ability to capture and demonstrate results to a variety of constituencies and to

promoting learning and enabling decision makers to make informed decisions and

plan strategically.’

Independent Evaluation Office

2015 – Year of Evaluation

Declared by EvalPartners and currently working on a UN resolution to have 2015 declared Year of Evaluation

Motto: Empowering nations and people through evaluation

19

Independent Evaluation Office

EvalYear

EvalYear, a catalyst to promote

demand and use of evaluation in policy-making

Inauguration of EvalYear at country level

Evaluation policy process at

country level

Promote national evaluation capacity building

Strengthening work with

parliamentarians and policy

makers (enabling

environment)

Strengthening stakeholders and

facilitate collaboration/ partnerships among them

20

Independent Evaluation Office

Concluding thoughts..

It is necessary to drive evaluations at many levels & ensure that all of the complexity and context is addressed.

Evaluation principles should not be lost in this pursuit Working across sectors and with partners helps build

support Evaluation should ultimately help to improve the lives of

people through ensuring development effectiveness. Professional and well capacitated evaluation functions

remain critical for this, and training such as IPDET is invaluable.

21

Thank you

United Nations Development Programme

Independent Evaluation Office220 East 42nd StreetNew York, NY 10017www.undp.org/evaluation