Role of Evaluation Units in fulfilling Evaluation Mandate

1

Pre-Conference for the International Year of Evaluation 2015Swiss Evaluation Society (SEVAL) / Geneva Evaluation Network (GEN)

Chaitali Chattopadhyay, WSSCC

Presentation Structure

I. WSSCC and its Evaluation Theory of Change

I. Case Study-Sanitation and Hygiene Thematic Window in partnership with International Initiative for Impact Evaluation (3ie)

II. WSSCC Evaluation role/functions: Reflections and way forward

2

3

• Founded in 1990 by the UN General Assembly resolution

• A global, multi-stakeholder membership and partnership organization

• Works with poor & marginalized people, their organizations, governments to improve sanitation and hygiene at scale

• A membership of over 3000 water, sanitation and hygiene (WASH) professionals from over 130 countries

• Governed by a Steering Committee of elected members from the membership constituency

• Hosted by the United Nations (WHO until 2009 and UNOPS since 2010)

• WSSCC’s main corporate priorities: Equality and Non-discrimination and Access and Use at scale.

Guiding principles

Spirit of collaboration,

facilitator & brokering a

collective leadership, neutral player

Policy advocacy, voices of people, justice, human

rights, most vulnerable and

marginalised people

4

WSSCC Organogram

5

6

Where do we work

A multi-stakeholder environment

Evaluation Mandate: The Theory of Change

Organizational and Sectoral learning-central to the Evaluation function

Accountability for the results

Committed in this MTSP

8

Evaluation: the journey so far…

Period covering 2000-2004 2005-2011 2012-2016

Raison d'être Corporate performance,

relevance, value for money assessment

Corporate performance, relevance, value for money assessment

Sector contribution, sector learning center stage , Policy

influencing plans, multi-sectoral approach, minimize

duplication

Evaluation Mandate

Donor sponsored Donor sponsored WSSCC /Organization driven

Funding support Donors Donors Embedded in the organisational budget

External quality assurance

mechanisms

No No Peer review/advisory mechanisms in place

Nature of Evaluation

External Evaluation External Evaluation External Evaluation (s)

Evaluation structure,

dedicated HR

No separate M&E unit No separate M&E unit Dedicated M&E unit 2 full time technical staff under

the NKM PM

9

Quick facts

10

Structure Not an independent Evaluation Office/ stand-alone Co-located- Networking, Knowledge Management Dept.

Maturity Relatively nascent, the M&E Unit- 2013

Policy Framework UNEG institutional partner, Not a full member No organizational Evaluation Policy

Eval Staff versus total organizational staff

Two full-time staff, NKM Programme Manager versus 41 staff

Monitoring and Evaluation functions

Still within the same unit, likely to be separated

Budget in the current MTSP

1% of overall organizational budget

How can we contribute towards an evidence- based culture and practice?

How can we go beyond the WASH Sector and make it a multi-disciplinary, multi-sectoral learning and enriching experience ?

Can we strengthen monitoring and evaluation of advocacy and policy influencing?

Process should be as important as the evidence

How can we contribute meaningfully towards the International Year of Evaluation?

11

Case study: The sanitation and hygiene thematic window for impact Evaluation

WSSCC & 3IE: A unique/complementary partnership

WSSCC

Global membership organization, spirit of collaboration, facilitator & brokering a collective leadership, Policy advocacy, voices of people and experience of managing a Global fund for sanitation and hygiene

3IE

Mandate to generate high-quality evidence to deliver better policies and program in order to improve lives

Cutting-edge technical expertise

Developing country focused grant portfolio

12

Evaluation Advisory Committee

Multi-disciplinary experts- Evaluation, WASH, Health/Reproductive health, Academia/ Research, Donor constituency

Provide technical support, minimise duplication, inter-agency collaboration, learning and sharing to collectively influence an evidence-building culture

Who are they?

13

Sam Bickle

UNICEF

Andrea E Cook

UNFPA

Dr Pavani Ram

Univ of Buffalo

Dr Bertha

Briceno, IADB

Johan Sundberg

SIDA

Impact Evaluations (2) Systematic Reviews (2)

Evaluating advocacy approaches in development

WSSCC Mid Term Review (MTR)

FACETS OF WORK

14

Features/strengths

Equity and Gender- Evaluation team composition

Independence with inclusion – Senior management in safeguarding independence

– process important- enhancing stakeholder involvement in evaluation

Utilization focused, results into action, management response

15

Ground realities…

• Evolving evaluation culture

• Limited systems of monitoring and data quality assurance

• Nearly 50% of our programmes do not have conventional results framework/log frame- challenging the evaluability

• Oversight/ management functions along with the core evaluation function

16

Forward looking vision

Member state driven- rather than agency driven- putting principles of aid effectiveness at the center-stage

Multi-agency/multi-sectoral learning, avoid duplication

Increase in joint evaluations

Use of evaluation to support evidence-based policy and decision-making

National evaluation capacity development- member states and the local governance level 17

What does this mean for WSSCC?

• Accountability for whom?

• Investing in evidence building

• Applying evidence to its own work

• Preparing for change/ Reinvigorating itself

18

THANK YOU

19