Evaluation in a Turbulent World Challenges, opportunities and innovation in evaluation practice

IDEAS Global Assembly11-15 April 2011

Evaluation of WASH PerformanceCase of Nepal

Amrit Rai: Evaluation in a Turbulent World/UKES/22-23Nov.2010

PRESENTATION CONTENTS COUNTRY’S BRIEF SANITATION SITUATION

Status Actors Approaches

ISSUES/CHALLENGES Result measurement Framework > NPC format, MfDR Policies Compliance> NPC, Ministries Programmes Compliance> Departments Projects > Line agencies, NGOs, Projects

SOLUTIONS Approach > CLTBCHC Institutional/Governance > Service Delivery-Accountability

CONCLUSIONS

Figure 1: Sanitation – Existing Situation and Targets

Source: Sanitation and Hygiene Master Plan 2010

Vision

Objective

Strategy

Policy/Working Policy

Programme

Project

Ministry of Physical Planning and Works

Ministry of Local Development

Department of Water Supply and

Sewerage

Department of Local

Infrastructure Development

and Agricultural

Roads

Rural Village Water

Resource Management

Project (Finnish)

Rural Reconstruction

and Rehabilitation Project (UK,

Japan)

Community Based Water Supply and Sanitation Project

(ADB)

Board/INGOs/NGOsWater Aid, NEWAH

Activities

75 Districts/3,915 Village Development Committees

Water supply and Sanitation Divisional

Offices

Regular activities

Regular activities

Rural Water Supply and Sanitation Project in

Western Nepal (Finnish)

Activities ActivitiesActivities Activities Activities

Logical Relationship of WSS Sector Development Approach

National Planning Commission

Plan, Strategy, Policy and Programme Formulation

Measuring Development Effectiveness

Strategy Policy Progra

mmeProjec

tObject

iveActivit

iesVision

Effectiveness

CoordinationHarmonizationComplianceEnforcementResource generation

Different approach/modalityDuplicationPriority

WASH Sector

“raising the living standard ...by making sustainable and equitable water supply and sanitation services available

WSS universal access by 2017

Programme Logic Model

Target : 60% (2010)Achieved: 43 %

Outputs

Efficiency

OutcomesImpacts

Agencies Business Plan

Result Framework

Ministry of Physical Planning and Works (MPPW)Department of Water Supply and Sewerage (DWSS) √

Department of Road √Department of Transport Management √ √Ministry of Local Development (MLD) √Department of Local Infrastructure Development and Agriculture Road (DoLIDAR)Ministry of Education √Ministry of Energy √ √Ministry of Agriculture and Cooperative √ √Department of Agriculture √Nepal PoliceMetropolitan Traffic Police Division √ √

Govt. Agencies Using Managing for Development Results (MfDR)Source: http://www.npc.gov.np/mfdr/?tour=pilot_agencies

Project/INGO/NGO

Water and Sanitation

Users Committees

(WUSC)

Users/CitizensService

Compact

Contractual relationship• Monitoring• Supervision• Compliance• Enforcement

Service delivery• Quantity• Quality• Cost• Time

Project/INGOs/NGOs can also listen to the voice of citizens but they hardly act on it for service standard improvement or to increase service coverage because they work under the given mandate as per contractual agreement between two countries or organizations. In a way they act as a whole sale agent or service provider of a national or foreign government agency or international bodies.

Direct Relationship

Service ProviderPeople

Compact

Development Agent

Less potentiality to mobilize local resources(VDCs, NGOs, other users committee, CBOs, donor supported projects and private sector… )

Donors prefer this model if

the country is in a conflict or

fragile situation .

Accountability Framework

District Development Committees

(DDC)

Water and Sanitation

Users Committees

(WUSC)

Users/CitizensService

Voice

Compact

Contractual relationship• Monitoring• Supervision• Compliance• Enforcement

Service delivery• Quantity• Quality• Cost• Time

Demand, grievances, complaints• Service Standards• Service financing• Service accessibility• Service coverage etc..

Service Provider fails to provide the service according to the contractual agreement when the corruption takes place between WUSC and DDCDirect Relationship

Client Power

State- Local Government

Service ProviderPeople

Compact

Inputs: contract /procurement managementPPP structuring

Inputs: Demand driven planning

Accountability Framework

Government & Line

Agencies

Water and Sanitation

Users Committees

(WUSC)

Users/CitizensService

Voic

e Compact

Contractual relationship• Monitoring• Supervision• Compliance• Enforcement

Service delivery• Quantity• Quality• Cost• Time

Demand, grievances, complaints• Service Standards• Service financing• Service accessibility• Service coverage etc..

Service Provider fails to provide the service according to the contractual agreement when the corruption takes place between WUSC and DDCDirect Relationship

Client Power

State- Local Government

Service ProviderPeople

Compact

Inputs: contract /procurement managementPPP structuring

Inputs: Demand driven planning

Accountability Framework

Parliament

District Development Committees

(DDC)

Water and Sanitation

Users Committees

(WUSC)

Users/CitizensService

Voice

Compact

Contractual relationship• Monitoring• Supervision• Compliance• Enforcement

Service delivery• Quantity• Quality• Cost• Time

Demand, grievances, complaints• Service Standards• Service financing• Service accessibility• Service coverage etc.. Direct Relationship

Client Power

State- Local Government

Service ProviderPeople

Compact

Accountability Framework

Key Features Community Level Total Sanitation (CLTS)

School Led Total Sanitation (SLTS)

Community Led Total Behavioural Change in

Sanitation & Hygiene (CLTBCHS)

Year 2003 2006 2009Main strategy • ODF declaration

• IPRA tools• appropriate low cost

toilet

• Child friendly WATSAN and ODF

• Community awakening• Partnership

• Ignition and triggering• Service governance• Total behavioural change• Negotiation for change

Support mechanism

• Subsidy for household toilets

• Partial support, cost sharing, community reward,

• Rewarding after ODF declaration

• TBC continuous process

Coverage targets • Community or cluster • School and its catchment area • Blanket VDCTools and techniques

• Specially IPRA tools for triggering communities

• Sensitization workshop and orientations

• IPRA,IEC materials

• IPRA, IEC, BCI, Triggering Tools

Strength • Communities/ clusters declared ODF

• School catchment areas• Child club • Local institutional support• Resource pooling

• Strong institutional setup• DDC lead ODF process• Wider sanitation choice• Resource affordability• Local resource mobilization

Limitation • Lack of post ODF institutional support and flexibility in financing

• Needs additional funds for water, toilet, hand washing

• Negotiation is necessary to declare TBC in H & S.

Structure Focus • Programme • Programme • Institution and ProgrammeResults • 74 ODF VDCs (9.25 ODF VDCs/year) • 29 ODF VDCs (14.5 ODF

VDCs/year)

•Policy Failure

•Strategy Failure

NATIONAL PLAN VS RESULTS

•Institutional Failure

PROGRAMME VS RESULTS

•Approach Failure

PROJECT VS RESULTS

CHAIN OF NEGATIVE DEVELOPMENT EFFECT