Water utilities performance assessment Multicriteria methodologies contributions

The International Conference on Adapting to Climate ChangeWater, waste and other local infrastructure

Sandra Tralhão Rita Martins

and João Paulo Costa

1

Agenda

1. Introduction2. Methodological approach

2.1. Regulation and performance evaluation2.2. Multiple Criteria Decision Aid Methodologies

3. Environmental sustainability of water supply services bulk operators in Portugal3.1. Identification and characterization of indicators3.2. Application and discussion of results3.3. Sensitivity analysis

4. Conclusions

2

1. Introduction

The idea of unlimited water resources meant that for centuries' water was considered an inexhaustible resource. However:

Significant increase in           population

Consumer habits changes

Significant increase in           population

Consumer habits changes

Significant increases in demand Significant increases in demand

reduction of water resources availability reduction of water resources availability

Climate changes 

Occurrence of more frequent extreme climatic events

Climate changes 

Occurrence of more frequent extreme climatic events

Changing drainage

Lowering of groundwater levels and contraction of lakes 

Changing drainage

Lowering of groundwater levels and contraction of lakes 

Irregularities of freshwater supply and decrease of the water quality 

Irregularities of freshwater supply and decrease of the water quality 

3

1. Introduction

The goal of this work is to assess the environmentalsustainability of water supply services bulk operatorsin Portugal.

The basis for the present research is the performanceevaluation model adopted by ERSAR.

However, our analysis allows to adjust the importancegiven to each criterion in a specific management periodaccording to the defined sustainability goals.

4

2. Methodological approach

• The methodological approach is supported by benchmarking toolsand indicators, used by many organizations at the internationallevel, and in Multiple Criteria Decision Aid Methodologies.

The problematics that fall within the decision‐making 

context 

Sorting

‐ Multiattribute Value Theory (MAVT) ‐ Outranking method ‐ ELECTRE TRI 

Ranking Multiattribute Value Theory (MAVT)

5

2.1. Regulation and performance evaluationThe need to regulate the quality of water supply services is consensual andbenchmarking tools are used by the most known regulators at theinternational level.

6

2.2. Multiple Criteria Decision Aid Methodologies

Multiattribute Value Theory (MAVT) is a compensatory method of additive aggregation that allows:

‐ To associate an overall score to an alternative‐ To build the complete ranking of the alternatives ‐ That a poor performance on a criterion may be compensated by high 

performance in another criterion

∑

is the overall value of alternative is the weight of criterion j

reflects the performance of alternative  for the criterion j7

2.2. Multiple Criteria Decision Aid Methodologies• The outranking ELECTRE TRI method is a multicriteria‐based outranking sorting

method which allows assigning alternatives into a set of predeterminedcategories.

• In the construction of the outrank relations the parameters of the criteriadefining the importance coefficients (wj), the performance of each alternative ina given criterion and the preference (pj), indifference (qj) and veto (vj) thresholdsare used.

Source: Adapted Mousseau et al. (1999)

Category 1

Category 2……

Category n…

8

3. Environmental sustainability of water supply services bulk operators in Portugal

• The designation of the watersupply sector in Portugaldepending on the type ofactivities carried out byoperators:

‐ Bulk water supply‐ Retail water supply

• The specific economic regulatorin Portugal is the Water andWaste Services RegulationAuthority ( ERSAR).

Distribution

Storage

Adduction

Elevation

Treatment

Abstraction

Bulk water su

pply

Retail water su

pply

9

Evaluation sheets – operator’s performance is exposed indicator to indicator.

Indicators of environmental sustainability.

10

3.1. Identification and characterization of the indicators

• The real water loss (RWL) indicator is used to assess the level ofenvironmental sustainability of the efficiency in use of environmentalresources. Is measured by the water losses by total length of pipes .

RWLLosses m3/year

Totallengthofpipes km ∗ 365

Good servicequality

Acceptable service quality

Unsatisfactory service quality

[0,0; 5,0] ]5,0; 7,5] ]7,5; +∞[

11

3.1. Identification and characterization of the indicators

• The fulfillment of water abstraction licensing (FWAL) indicator allows theevaluation of the environmental sustainability of service, in terms of thesafety of water abstractions and the environmental protection.

FWAL / /

∗ 100

Good servicequality

Acceptable servicequality

Unsatisfactory service quality

100 ]90,0; 100[ [0,0; 90,0[

12

3.1. Identification and characterization of the indicators

• The standardised energy consumption (SEC) indicator intends to assessthe level of efficiency in the use of environmental resources, taking intoaccount the appropriate use of energy resources.

• It is defined as the average power consumption of normalized pumpingfacilities.

SEC / / ∗

Good servicequality

Acceptable service quality

Unsatisfactory service quality

[0,27; 0,40] ]0,40; 0,54] ]0,54; +∞[

13

3.2. Application and discussion of resultsAlternatives Real water

losses

Fulfillment of water abstraction licensing

Standardisedenergy

consumptionAdSAndré 8,42 54,1 0,49AdTMAD 1,53 98,04 0,4

AdAlgarve 4,73 60,42 0,38AdCentro 1,22 29,37 0,43

AdCAlentejo 3,05 83,01 0,5AdDPaiva 11,27 94,39 0,37

AdMondego 10,64 95,73 0,39AdNoroeste 1,25 98,3 0,3

AdNAlentejano 2,94 94,39 0,49AdOeste 4,26 0 0,47AdVouga 1,36 50,21 0,4AdZCoa 2,77 30,51 0,37

AdPAlentejo 4,4 35,36 --EPAL 34,88 100 0,38ICOVI 4,52 0 0,69

Real water losses Fulfilment of the water abstraction licensing

Standardised energy consumption

b2 5,00 100,00 0,40 b1 7,50 90,00 0,54

Performance matrix based on the evaluation sheets of the quality of service published by ERSAR for the year 2013. 

Standard profiles14

3.2. Application and discussion of results ‐ MAVT

Global value Good service quality

Acceptable service quality

Unsatisfactory service quality

0,5877 0 0 1 AdSAndré 0,7929 1 1 1 AdTMAD 0,6852 0 1 1 AdAlgarve 0,6144 0 0 1 AdCentro 0,6966 0 1 1 AdCAlentejo 0,7194 0 1 1 AdDPaiva 0,7183 0 1 1 AdMondego 0,8433 1 1 1 AdNoroeste 0,7299 0 1 1 AdNAlentejano 0,4995 0 0 1 AdOeste 0,6781 0 1 1 AdVouga 0,6332 0 0 1 AdZCoa 0,3325 0 0 1 AdPAlentejo 0,5372 0 0 1 EPAL 0,3929 0 0 1 ICOVI

• Weigths ( withthesameimportanceadjusted by scale factors

• Global profiles adjusted by scale factors:

b1= 0,770495827b2 = 0,657142857

15

3.2. Application and discussion of results ‐ MAVT

• From the application of the MAVT method results the ranking of theenvironmental sustainability alternatives.

• The highest overall value function obtained is 0,8433 ("AdNoroeste“) andthe second is 0,7929 ("AdTMAD“)

• Most operators get unsatisfactory performance.

16

3.2. Application and discussion of results – ELECTRE TRI

Good service quality

Acceptable service quality

Unsatisfactory service quality

0 0 1 AdSAndré 1 1 1 AdTMAD 1 1 1 AdAlgarve 0 0 1 AdCentro 0 1 1 AdCAlentejo 0 1 1 AdDPaiva 0 1 1 AdMondego 1 1 1 AdNoroeste 0 1 1 AdNAlentejano 0 0 1 AdOeste 0 1 1 AdVouga 0 0 1 AdZCoa 0 0 1 AdPAlentejo 0 0 1 EPAL 0 0 1 ICOVI

Good service quality Acceptable service quality Unsatisfactory service quality

•Threshold ‐ Indifference ( )=0‐ Preference  ( )=0 ‐ Veto  ={10,55, ∞}

•Weigths( )={0,33;0,33;0,34}

• Standard profiles• Cutting level =0,5 

17

3.2. Application and discussion of results – ELECTRE TRI

• In this case the results affect the same operators ("AdTMAD“ and"AdNoroeste“ of the MAVT) and "AdAlgarve" to the goodenvironmental sustainability performance category .

• About 46.6% of the operators are allocated to performance category"unsatisfactory" and 33.3% to the "acceptable".

• For > = 0,57 the classification gets significantly worse and allocates80% of operators to the "unsatisfactory” performance category (in thisscenario are obtained similar results from the application of MAVT).

18

3.3. Sensitivity analysis• With the two methods use the results reveal themselves more favorable if a

higher weight is assigned to water loss.

• With the two methods the worst case of environmental sustainability is theone giving a higher weight to the water abstraction licensing criterion.

• increasing the cut off level decreases environmental sustainability obtainedwith ELECTRE TRI method (Cutting level > =0,69)

• The scenarios obtained by changing the weights assigned to the criteria aremore favorable when the water loss and the energy efficiency criteria areassigned a weight greater than or equal to 0.5.

19

3.3. Sensitivity analysisGood service

qualityAcceptable

service qualityUnsatisfactoryservice quality Operators

0 0 1 AdSAndré

0 1 1 AdTMAD

0 0 1 AdAlgarve

0 0 1 AdCentro

0 0 1 AdCAlentejo

0 0 1 AdDPaiva

0 0 1 AdMondego

0 1 1 AdNoroeste

0 1 1 AdNAlentejano

0 0 1 AdOeste

0 0 1 AdVouga

0 0 1 AdZCoa

0 0 1 AdPAlentejo

0 0 1 EPAL

0 0 1 ICOVI

For cutting level >= 0,69 in ELECTRE TRI

20

Good servicequality

Acceptableservice quality

Unsatisfactoryservice quality Operators

0 0 1 AdSAndré

1 1 1 AdTMAD

1 1 1 AdAlgarve

0 1 1 AdCentro

1 1 1 AdCAlentejo

0 0 1 AdDPaiva

0 0 1 AdMondego

1 1 1 AdNoroeste

1 1 1 AdNAlentejano

0 1 1 AdOeste

1 1 1 AdVouga

1 1 1 AdZCoa

0 0 1 AdPAlentejo

0 0 1 EPAL

0 0 1 ICOVI

The most favorable scenario, in ELECTRE TRI  is the one where it is attributed to the water loss indicator the weight the  0.9.

3.3. Sensitivity analysis

21

4. Conclusions• The exploration of the potential application of MCDA methodologies for

the assessment of performance in sorting and ranking problematicsshowed they can be appropriate to this decision problem.

• The results, although slightly more favorable in ELECTRE TRI, reveal a highnumber of operators with unsatisfactory sustainability and environmentalperformance.

• The two methods highlight the good performance of the "AdTMAD" and"AdNoreste".

• The application of the additive aggregation method‐MAVT, and of theprevalence ELECTRE TRI method resulted in similar results, since thecompensatory nature of MAVT is counteracted by the veto threshold inELECTRE TRI.

22

Water utilities performance assessment Multicriteria methodologies contributions

Thank you

Sandra Tralhão1, Rita Martins2, João Paulo Costa31 Faculty of Economics, University of Coimbra, stralhao@gmail.com

2 Faculty of Economics and CeBER/GEMF, University of Coimbra, rvmartin@fe.uc.pt3 Faculty of Economics and INESC Coimbra, University of Coimbra, jpaulo@fe.uc.pt

References1. Armstrong, M., Cowan, S., & Vickers, J. (1999). Regulatory Reform – Economic Analysis and British Experience. Cambridge: The MIT Press.2. Artley, W., & Stroh, S. (2001). Establishing an Integrated Performance Measurement System (2). Performance‐Based Management Special Interest Group.3. Belton, V., & Stewart, T. (2002). Multiple Criteria Decision Analysis: An Integrated Approach. Springer Science & Business Media.4. Church, J., & Ware, R. (2000). Industrial Organization ‐ A Strategic Approach. United States: Irwin McGraw‐Hill.5. Danielson, M., & Ekenberg, L. (2016). The CAR Method for Using Preference Strength in Multi‐criteria Decision Making. Group Decision and Negotiation, 25, 775–797.6. Dezert, J., & Tacnet, J.‐M. (2012). Soft ELECTRE TRI outranking method based on belief functions. IEEE, 607 ‐ 614.7. ERSAR, & LNEC. (2012). Guia técnico nº 19 ‐ Guia de avaliação da qualidade dos serviços de águas e resíduos prestados aos utilizadores ‐ 2.ª geração do sistema de avaliação. Entidade 

Reguladora dos Serviços de Águas e Resíduos.8. Figueira, J., Roy, B., & Mousseau, V. (2005). ELECTRE Methods. In J. Figueira, S. Greco, & M. Ehrgott, Multiple Criteria Decision Analysis ‐ state of the arte surveys (pp. 133‐162). Boston: 

Springer Sciense.9. Grupo de Trabalho Operacional e Comissão de Acompanhamento do PENSAAR 2020. (2014). PENSAAR 2020 ‐ Uma nova Estratégia para o Setor de Abastecimento de Água e Saneamento 

de Águas Residuais, 2. Plano Estratégico de Abastecimento de Água e Saneamento de Águas Residuais. Portugal.10. Keeney, R. (2007). Enquadramento de decisões de política pública. In C. Antunes, & L. Dias, Decisão: Perspetivas interdisciplinares (pp. 173‐210). Coimbra: Imprensa da Universidade de 

Coimbra.11. Keeney, R., & Raifa, H. (1976). Decisisons with multiple objectives: prefrences and value trade‐offs. New York: John Wiley & Sons.12. Lourenço, R., & Costa, J. (2004). Using ELECTRE TRI outranking method to sort MOMILP nondominated solutions. European Journal of Operational Research, 153, 271–289.13. Marques, R. (2005). Regulação de Serviços Públicos. Lisboa: Edições Sílabo, Lda.14. Martins, R. (2007). Regulação económica no setor das águas ‐ Promoção da concorrência e sustentabilidade tarifária. Tese de Doutoramento. Faculdade de Economia da Universidade de 

Coimbra.15. Melo‐Baptista, J. (2014). Uma abordagem regulatória integrada (ARIT‐ERSAR) para os serviços de águas e resíduos. Entidade Reguladora dos Serviços de Águas e Resíduos (ERSAR).16. Mousseau, V., Slowinski, R., & Zielniewicz, P. (2000). A user‐oriented implementation of the ELECTRE‐TRI method integrating preference elicitation support. Computers & Operations 

Research, 27, 757‐777.17. OCDE. (2015). OECD Principles on Water Governance ‐ Water Governance Initiative. OCDE.18. Parlamento Europeu e Conselho da União Europeia. (2010). Directiva 2000/60/CE. Jornal Oficial das Comunidades Europeias.19. Riabacke, M., Danielson, M., & Ekenberg, L. (2012). State‐of‐the‐Art Prescriptive Criteria Weight Elicitation. Hindawi Publishing Corporation ‐ Advances in Decision Sciences.20. Roy, B. (1996). Multicriteria methodology for decision aiding. Kluwer Academic Plublishers.21. Shleifer, A. (1985). A theory of yardstick competition. Rand Journal of Economics,16 (3), 319,327.22. Tundisi, J. (2003). Água no século 21: enfrentando a escassez. RiMa, IIE.23. Vincke, P. (1992). Multicriteria decision‐aind. Bruxelles: Jonh Wiley & Sons, Inc.24. Watson, S., & Buede, D. (1987). Decision synthesis. The principles and practice of decision analysis. Cambridge: Cambridge University Press.25. Zopounidis, C., & Doumpos, M. (2002). Multicriteria classification and sorting methods: A literature review. European Journal of Operational Research 138, 229–246. 24