1

How to make monitoring a common and lasting practice ?

Lessons from El Salvador, Honduras and ParaguayIRC Lunch seminar, 22 May 2013

Key messages

• Monitoring important to improve sustainability, by providing information to target post-construction support to service providers

• Many monitoring systems often suffer sustainability problems themselves, therefore there is need to identify such risks and mitigation measures

• This can be done through a structured approach for defining institutional arrangements through a multi-stakeholder process

Background• IDB re-entering rural WASH, but found:

– Many of the systems they had supported before, no longer functioning adequately

– Few countries with up-to-date inventories of rural WASH assets

• Triggering interest in monitoring sustainability of rural water supplies for:

– Targeting post-construction support

– Informing investment plans

• Need to address sustainability of monitoring systems themselves, by having clear institutional framework and costing of them

Honduras: background and approach• Used to have reasonably well-functioning rural water information system (SIAR), but got out-of-date

quickly after donor support ended

• Joined regional monitoring initiative (SIASAR)

• Implying a move from an organizational to a sector monitoring system

– Opening up access to all sector stakeholders

– Meeting needs of different stakeholders

– Decentralising and distributing data collection

• Structured approach to develop this institutional framework and costing

Honduras: key issues

• Data collection: by centralised agency or by municipalities and service providers

• Validation of data: spot-checks, sieving out obvious errors and through open access of information

• Processing: need for careful calibration of algorithms

• Analysis and interpretation: the weakest link, with limited capacity at decentralised level

Paraguay: main characteristics

• Coverage figures rural/urban water 66/99 and for sanitation 40/90 (JMP -2012)

• No sector leadership and highly fragmented sector (institutions and systems)

• Process lead by SENASA- capacity is weak and centralized in Asunción

• Establishment of a multi-stakeholder working group

• Involvement in design of the system (purpose, scope)

• Sub-groups in development of the indicators (service, service provider, TA)

• Development of the technological platform

by Geoinformatica – piloted in 2011

Paraguay: highlights

• Work on design of the system with multi-actor working group:

– Allowed the introduction of SDA as opposed to the infrastructure and implementation focus in the sector

– Helped to arise awareness on the need for one single framework for monitoring WASH services- ( alignment between systems and collaboration among actors- institutional arrangements)

• Work on the indicators: heated debate on norms and benchmarks for service delivery

• Agreement of a staggered process in terms of comprehensiveness of the system and application at scale

• Critical are clarity on management and governance of the system (need of allocation of dedicated resources)

El Salvador

• Centralised sector with one national utility; community-based management unregulated

• Wish to get better understanding of status of rural water, with initial focus on doing a first inventory of assets and services provided

– Indicator development around service levels, service provider performance and access to support

– Algorithms to reach an overall score (A to D)

• Triggered interest in and need for post-construction support mechanisms to act upon results

Key elements of the approach

• Defining monitoring systems as set of procedures for carrying out

monitoring:

– Objectives

– Processes

– Institutional arrangements

– Information system

• Critically assess the capacity to use and maintain it:

– Financing framework

– Human resources to fulfil responsibilities

– Institutional ownership and governance

• Multi-stakeholder process, with leadership by main government agency

– even though not always clear who that is

Overview of the approach

Step 

StakeholderData collection Processing Analysis Reporting

Identifying corrective actions

Service providersOn-going but unstructured

Without standard procedure or tool

Without standard procedure or tool

Annual reports to users 

Some decision making tools for water quality and administration

Municipal Association of Water Committees

On-going but unstructured

Sometimes, but without standard procedure or tool

Without standard procedure or tool

Unknown Unknown

Operation and Maintenance Technician

Using standard tool. Demand-based and depending on resources

Using SIAR Using SIARTo the service provider and national utility

Based on standard set of typical corrective measures

Regulation and Control Technician

Using standard toolAs above, but using other information system

By national regulatorReports on website of national regulator

National regulator informs municipality to take action

Environmental Health Technicians

Using standard tool, but limited Resource

Data provided to Regional Health Secretariat

Unknown Unknown Unknown

NGOs and projectsDetailed assessments based on project needs

Based on own criteria Based on own criteria InternalFeasibility assessment of Project

Tools: mapping current monitoring activities

Institution

Roles

System manager Data collector Validator

Information user

PAT

Technical assistance provider (SANAA)

Policy making body (CONASA)

Regulator (ERSAPS)

Implementing agency (FHIS)

Health Secretariat

Finance and Planning Secretariat

Association of Municipalities of Honduras

Municipalities

Associations of Water Committees

Water Committees

NGOs

Tools: Matrix to definition roles

Tool: Matrix for defining the new institutional arrangements

• Broken down by steps in monitoring

• Phases in the development of the monitoring system

Data collection

Validation ProcessingPublishing of

resultsAnalysis

Identifying corrective measures

Self-reporting by service providers

In municipal Water Roundtables, bringing together municipal officials and water committees. ANDA (national utility) revises information to identity obvious errors and uploads to database

Automatized but under supervision of ANDA

ANDA makes national synthesis report. Municipalities make local reports based on the results from database, where needed supported by ANDA

Municipal Water Roundtables do this jointly

Municipal Water Roundtables do this jointly, supported by ANDA or other PATs

Tools: Costing and responsibilities for financing

• Detailed budgeting of all steps in monitoring

– Differentiated between baseline and regular monitoring

– Quantifying all costs, including hidden costs, e.g. time of communities and local government staff

• Verification through rounds of piloting

• Based on this, agree on financial agreements

• Check whether the costs can be met feasibly by different bodies

• Iterations – to adjust budgets or ambitions

• Continuous monitoring of the costs of monitoring in the roll-out

Results of costing

• Main costs are in data collection, but don’t underestimate costs of analysis

El Salvador Honduras Paraguay

Baseline 0.39 0.24-0.34(actual)

0.09

Regular updating

0.11 + 0.08 0.23

Lessons learnt from applications

• Build on existing monitoring practices even if they are incomplete or imperfect

• Decentralise most of the steps in monitoring but ensure sufficient support to local governments

• Gradual approach to the comprehensiveness of the scope of the monitoring

• Phased approach to reach country-wide scale, including pilots to adjust information system and create ownership within sector

• Costing is important tool in assigning responsibilities and assessing risks

For discussion

• How effective can monitoring WASH service delivery be in a sector

dominated by an implementation and infrastructure approach?

• How to overcome the weak institutional capacity at national and

decentralised levels for ensuring continuous WASH monitoring?