ISS-EWATUS D7.1 PLAN OF VALIDATION AND EVALUATION · This project has received funding from the European Unions Seventh Framework Programme for research, technological development

This project has received funding from the European Union’s Seventh Framework Programme for research, technological development and demonstration under grant agreement no [619228]

ISS-EWATUS FP7-ICT-2013-11 Page 1 of 38 D7.1 Plan of validation and evaluation January 2016

Grant Agreement number: 619228

Project Acronym: ISS-EWATUS

Project Full Title: Integrated Support System for Efficient Water Usage and Resources

Management

Funding Scheme: Collaborative project

Period covered: from 01/02/2015 to 31/01/2016

Name, title and organisation of the scientific representative of the project's coordinator:

Dr Wojciech Froelich, Dr Ewa Magiera, University of Silesia, Poland

Tel: +48 510294043

Fax: +48 32 2918283

E-mail: [email protected], [email protected],

Project website address: http://issewatus.eu

ISS-EWATUS

D7.1 PLAN OF VALIDATION AND EVALUATION

mailto:[email protected]

http://issewatus.eu/



Copyright

© Copyright 2014-2017 The ISS-EWATUS Consortium

Consisting of:

Organisation Name Short Name Country

UNIWERSYTET SLASKI US Poland

INSTYTUT EKOLOGII TERENOW UPRZEMYSLOWIONYCH IETU Poland

Rejonowe Przedsiębiorstwo Wodociągów i Kanalizacji w Sosnowcu Spółka Akcyjna

RPWiK Poland

LOUGHBOROUGH UNIVERSITY LU United Kingdom

BRUNEL UNIVERSITY BU United Kingdom

UNIVERSIDAD PABLO DE OLAVIDE UPO Spain

CENTRE FOR RESEARCH AND TECHNOLOGY HELLAS CERTH Greece

Dimotiki epixirisi Ydreusis - apoxeutesis Skiathou DEYASK Greece

DOTSOFT OLOKLIROMENES EFARMOGES DIADIKTIOY KAI VASEON DEDOMENON AE

DOTSOFT Greece

STICHTING VU-VUMC VU/VUmc Netherlands

Disclaimer

All intellectual property rights are owned by the ISS-EWATUS consortium members and are protected by the applicable laws. Except where otherwise specified, all document contents are: “© ISS-EWATUS Project - All rights reserved”. Reproduction is not authorised without prior written agreement.

All ISS-EWATUS consortium members have agreed to full publication of this document. The commercial use of any information contained in this document may require a license from the owner of that information.

All ISS-EWATUS consortium members are also committed to publish accurate and up to date information and take the greatest care to do so. However, the ISS-EWATUS consortium members cannot accept liability for any inaccuracies or omissions nor do they accept liability for any direct, indirect, special, consequential or other losses or damages of any kind arising out of the use of this information.



History

Version Author Date Status

1.0 Ewa Magiera 12-2015 First Draft

2.0 Workpackage leaders From 12-2015 to 02-2016

Contribution of all workpackage leaders

3.0 Ewa Magiera, Wojciech Froelich

31-03-2016 Final version



Executive summary

The present document is a deliverable of the ISS-EWATUS project, funded by the European Commission’s Directorate-General for Communications Networks, Content & Technology (DG CONNECT), under its 7th EU Framework Programme for Research and Technological Development (FP7).

The document describes the validation and evaluation plan of the ISS-EWATUS project. To effectively describe and proceed with the validation and evaluation, we firstly need to define both of these notions. We also provide a timeline of the considered validation and evaluation. Furthermore, we relate the objectives of the validation and evaluation to the tasks of the Description of Work (DoW).

According to the requirement imposed by the DoW, we determine the categories of end-users of ISS-EWATUS. These categories of users are related to the software and hardware components of the implemented system. We also describe the heterogeneous environmental conditions (two different climatic zones) in which the validation and evaluation will take place. We illustrate the current state of ISS-EWATUS installations at Skiathos (Greece) and Sosnowiec (Poland).

The first challenge addressed when preparing the validation and evaluation plan was to enable quantitative evaluation of the project objectives. For this purpose we have defined a set of Key Performance Indicators (KPI). The KPIs have been related to the objectives imposed by the DoW. The formalisation of the evaluation process by the proper definition of the KPI is the first crucial element for the planned evaluation of the project outcomes.

After defining the set of KPIs, we have related them to the modules implemented as parts of the ISS-EWATUS. In this way, the evaluation of the project outcomes will be made in a consistent, integrated manner. By the calculation of the KPIs, it will be possible to evaluate the achievement of every objective of the project from two different perspectives. First, it will be possible to evaluate the extent to which each particular module contributes to the achieved effect. Secondly, the achievement of each imposed objective will be evaluated as a common goal, independently of the considered module.



Table of Contents 1.Introduction .......................................................................................................................................... 6

2.Objectives of the validation and the evaluation .................................................................................. 9

3.Categories of end-users of the ISS-EWATUS project .......................................................................... 11

4. Validation and evaluation places....................................................................................................... 12

4.1 Household level ........................................................................................................................... 12

4.1.1 Case study in Poland ............................................................................................................. 12

4.1.2 Case study in Greece ............................................................................................................ 13

4.2 Urban level .................................................................................................................................. 13

4.2.1 Case study in Poland ............................................................................................................. 14

4.2.2 Case study in Greece ............................................................................................................ 15

5.Project objectives and Key Performance Indicators (KPI) .................................................................. 17

5.1 Household level ..................................................................................................................... 17

5.1.1. Objective H1 – the increase of the awareness of water consumption ............................... 17

5.1.2 Objective H2 – the reduction of water consumption ........................................................... 18

5.1.3 Objective H3 – the inducement and the reinforcement of water saving behaviour of

consumers ..................................................................................................................................... 19

5.2 Urban level ............................................................................................................................ 19

5.2.1 Objective U1 – the reduction of leakages at municipal level ............................................... 19

5.2.2 Objective U2 - the reduction of peaks in water and energy distribution............................. 21

6.Procedure of the validation and evaluation of the ISS-EWATUS ....................................................... 23

6.1 Decision support system for households .................................................................................... 23

6.2 Decision support system for municipal water company ............................................................. 24

6.3 Social-media platform ................................................................................................................. 31

6.4 Adaptive pricing model. .............................................................................................................. 33

6.5 Evaluation .................................................................................................................................... 36

Conclusions ............................................................................................................................................ 36

References ............................................................................................................................................. 37



1.Introduction This document describes the validation and evaluation plan of all ISS-EWATUS modules. The structure of those modules is depicted in Figure 1.

Figure 1. Structure of the ISS-EWATUS

At household level, ISS-EWATUS proposes a low cost, mobile-device oriented set of tools supporting households in water conservation. The ISS-EWATUS makes users aware of their water consumption by providing near real-time access to their household water meters. On the basis of data gathered individually for every household, a mobile decision support system gives advice regarding water-saving behaviour to households. The Decision Support System (DSS) at household level is an information system designed to promote efficient water usage among residential consumers The home Wi-Fi system sends data collected from smart water meters to the remote server which records the water flow rate and water temperature associated with each appliance.

A mobile device provides access to a novel application which offers feedback on the household’s water use. The application allows users to view their household’s water consumption, broken down by appliance, across the past 24 hours or at a daily, weekly or monthly level. Users can set themselves a target for reducing their overall water consumption and the app will give them feedback on their progress towards this goal. A key component of the DSS is a news function which provides personalised information on saving water around the home. Bespoke tips are generated in response to the household’s recent, as well as predicted, water consumption. The graphical format of the DSS display has been designed to appeal to children as well as adults. To complement the automated feedback, a water diary function is provided to encourage households to come together and identify the water consumption associated with individual household members. An additional function uses information on a user’s showering and laundry habits, and household appliance efficiency, to highlight where a consumer’s water use practices do not align with their level of environmental concern.

At the urban level, the main goal of the ISS-EWATUS is to reduce water leaks within the water delivery system by adaptive-to-demand control over the pressure in the water delivery system.



Management of water demand is done via applications that will use data acquired from measurement devices installed in the water distribution network. The collected data are used to perform a consumption pattern analysis to provide evidence for leaks and trigger alerts; they are also used to predict the development of future water consumption, e.g., based on the historical consumption of water or other parameters. The urban DSS will help water companies to a) quantify leaks and suggest emergency actions, b) assess demand in the medium and long-term and c) manage demand through an optimal balance between supply and demand measures.

A social-media platform (SMP) (watersocial.eu) is a part of the ISS-EWATUS. It is able to support the promotion of water efficiency in a holistic approach. This includes its impact on local, national and international levels across Europe and its target audiences of water stakeholders at different levels of individuals, households, green NPOs, water managers, researchers and policy makers. The SMP aims to ease the communication and the creation of relationships between stakeholders and to produce a sustainable impact for the communities involved. Apart from supporting mainstream social networking activities such as sharing, communications, being friends, asking and answering, discussion. The SMP allows users to share water tips and photos under different environmental scenes and the share content can be pinned on a global map. Gamification enables the entire SMP to be used as a platform with gaming elements, which involve game tasks, competition and rewards. The game tasks can be any user tasks on the social networks or any water use related offline activities such as recording water-use activities. Each of these user tasks can be rewarded upon accomplishment.

The adaptive pricing system is a module for policy makers to assess the impact of pricing schemes. It allows stakeholders to assess the impact of pricing on water consumption and total revenues. In doing so, it is important to evaluate the fairness of pricing schemes by identifying the households that are affected most as well. The module can be used to design pricing schemes that create incentives to reduce the water consumption while maintaining revenues, or to increase revenues while maintaining the water consumption levels. The module identifies how difficult these objectives are to achieve and is able to generate optimal pricing schemes to satisfy predefined criteria. The module is thus an important tool for long-term strategic water management that is relevant for stakeholders that define water policies.

Let us define first the notions of validation and evaluation that will be used in this document.

Validation is the process of determining whether the stated purpose of the system was achieved. Validation means building the right system. A combined definition of verification and validation of software, is the analysis and testing to determine that it performs its intended functions correctly, to ensure that it performs no unintended functions, and to measure its quality and reliability. The simplicity and completeness of definition of validation in relation to simulation is attractive: “the adequacy of the model as a mimic of the system which it is intended to represent” (Sojda, 2004).

Evaluation of the outcomes of the ISS-EWATUS is understood as calculating the set of key performance indexes (KPIs) and checking if the obtained values reached the planned assumptions. The main goals of the evaluation are 1) an assessment of the achievement of project objectives, 2) estimating users’ satisfaction from the project outcomes, 3) impact assessment.

Figure 2 presents the timeline of the validation and evaluation of the ISS-EWATUS outcomes.



TIME OF THE COMPLETION – from 25 till 36 Month

VALIDATION EVALUATION IMPACT ASSEMENT

Figure 2. The timeline of the validation, the evaluation and impact assessment.

In the first part of this document we remind the assumptions of the project. We describe its expected outcomes together with the detailed objectives of the validation and evaluation. In Section 1, we introduce the overall idea of the project. In Section 2 we recall the requirements imposed by the DoW. The requirements are related to the tasks of the corresponding Workpackage 7 (DoW Part A, page 31). Section 3 recalls the validation places assumed in the project. In Section 4 we describe validation methodology related to the objectives of the project. In Section 5 Key Performance indicators are defined. The relationship of the project objectives to the KPIs is defined in Section 6.

Finally, we present the schedule of the planned works.



2.Objectives of the validation and the evaluation

Validation, evaluation and impact assessment of the project outcomes will be carried out according to the requirements stated in the description of Work package 7. The plan of work related to validation and evaluation of ISS-EWATUS is imposed on page 31 of the DoW (part A). This plan of WP7 consists of eight tasks, which are presented below.

Task 7.1 Plan of the validation and evaluation process

Categories of end-users participating in the validation process will be selected. Trials will be described in detail for all subsystems of ISS-EWATUS. All the necessary procedures concerning set-up, design, duration, main outcome measures and methodologies involved during the collection and analysis of the results will be determined. The set of measurement variables that will be used for the estimation of water conservation and the reduction of seasonal peaks of water and energy distribution loads will be described. Furthermore, the definition of other measures regarding increased water-saving awareness will be proposed.

Task 7.2 Setting up technological platform for the validation processes

The following actions will be taken in the sites premises before starting the validation phase:

• testing of modules in the validation context,

• final acceptance testing for the validation process,

• integration testing of all the ISS-EWATUS components to ensure uniformity, performance and reliability of the validation process. Issues that may arise will also be directly resolved in this task.

Task 7.3 Validation of the decision support system for municipal water company

In this task, it will be demonstrated how all network information is transmitted and accessed by the appropriate water stakeholders, with a user-friendly interface issuing warning signs for the decrease of water pressure, etc. The validation process will be run according to the Validation Plan and will cover an appropriate interval of time to summarise the results for the different seasons.

Task 7.4 Validation of the decision support system for households

The validation will be carried out at 10 households in Greece and 10 households in Poland where the pilot installation is performed. The validation process will be mainly focused on the comparison of water consumption before and after the ISS-EWATUS intervention strategies are implemented. A baseline of water consumption will be considered as a starting point. Applying various intervention strategies separately or together will generate the difference in water consumption. The validation will ascertain which intervention strategy generates most impact and how to apply the strategies.

Task 7.5 Validation of the social-media platform

This task will need a much larger involvement of water stakeholders and end users. Three pilots for the validation purposes will be released. The first one will be tested by the members of the consortium, and broadcast in limited availability, in order to gather more ideas on the requirements. The second prototype will be used by a larger number of users, that could comment on its usefulness more extensively, report broken features, and so on. The third prototype will be a collation of internal and external feedback. This task will require the participation of experts not only in the field of ICT but also in water resources management.



Task 7.6 Validation of adaptive pricing

The adaptive pricing module is based on models, the input for which is based on consumer behaviour data. The objective of this task is to generate predictions on water consumption in terms of changing the water tariffs (pricing schemes) and to compare them with baseline scenario. The validation process will be run according to the following criteria:

possible drop of water consumption as a result of changing the water tariffs,

social affordability of predicted tariffs,

financial effect of proposed changes for water providers,

consistency and compatibility of proposed changes with EU law (user pays principle).

Task 7.7 Indicators for measurement of impact

The objective of this task is to precisely define all quantitative measures for the evaluation of the outcomes of ISS-EWATUS. This task will produce several aggregative indicators for the assessment of water conservation and for the reduction of peaks of the water and energy distribution loads.

Task 7.8 Evaluation of the ISS-EWATUS project

The results of the project will be evaluated on different levels using quantitative measurable indicators and qualitative measures. The quantitative evaluation will be based on the indicators that will be defined in task 7.7.



3.Categories of end-users of the ISS-EWATUS project

As the validation will be run at household and city levels, engagement of different categories of system users will be required. To put through the validation, the following groups of users will be invited to participate in the validation process:

1. The members of households in Greece and Poland, where the installations of ISS-EWATUS project were implemented. Owing to the installation of smart sensors and meters, 40 households have access to the ISS-EWATUS system. All of them will be invited to validate ISS-EWATUS. But there is a serious fear that some of them may refuse to participate for various reasons.

2. Employees of two municipal water companies in Sosnowiec and Skiathos. Because RPWiK (Sosnowiec) and DEYASK (Skiathos) are the partners in the project, the validation process should be conducted smoothly and efficiently.

3. Users of the portal (http://watersocial.org). The dedicated to watersocial.org dissemination plan has been elaborated to ensure the appropriate number of portal users i.e. participants in the validation process. The Figure 3 presents the taxonomy of the users.

ISS-EWATUS USER

HOUSEHOLD WATER USER

MUNICIPAL WATER COMPANY EMPLOYEE

EMPLOYEE OF RPWiK(Water company

in Sosnowiec, Poland)

EMPLOYEE OF DEYASK(Water company

in SKiathos, GREECE)

HOUSEHOLD WATER USER FROM SKIATHOS (Greece)

HOUSEHOLD WATER USER FROM SOSNOWIEC (Poland)

SOCIAL-MEDIA PLATFORM USER

Figure 3. ISS-EWATUS validation and evaluation users’ taxonomy



4. Validation and evaluation places

Validation and evaluation of the decision support system will be conducted at two different geographical places (Poland and Greece) at household and city level. The range of validation of the social media platform will be only dependent on where the users involved in validation process live.

4.1 Household level

To the initial plan involved the installation of water consumption monitoring systems in 10 households in Sosnowiec (Poland) and 10 in Skiathos (Greece), where the decision support system for households is being demonstrated. However, reviewers of the proposal suggested adding more demonstration sites and for that reason the number of water consumption monitoring systems which have been installed was increased from 10 to 20 at each demonstration site. It is worth mentioning that each of 40 installations monitors hot and cold water consumption in near-real time.

There are two kinds of monitoring systems used in the project: the first is based on devices designed and provided by the Loughborough University as a part of the ISS-EWATUS (LU monitoring system) and the second is based on smart water meters provided by the Apator Company (Apator monitoring system). Both monitoring systems transmit data in near real time to the ISS-EWATUS spatio-temporal database to make them available for further processing and use in the household DSS.

Both systems have different advantages and disadvantages and the purpose of using two different systems was to ensure that the DSS is capable of using different data providers, i.e. it is not limited to one type of water consumption monitoring system.

The LU monitoring system provides more accurate data of higher temporal resolution (0,5 minute). The system, thanks to a link to the household Wi-Fi router, enables to see the real-time water consumption data via the DSS. The LU system is, however, more vulnerable to damage (it is installed on pipe connections in easily reachable places).

On the other hand, the Apator system provides less accurate data. The Apator monitoring system involves the cutting of pipes in walls which is a big disadvantage. Howeverwhen the system is installed, there is a negligible risk of damage.

Details on both monitoring systems can be found in the Deliverable 2.2 of the ISS-EWATUS project.

4.1.1 Case study in Poland

There are 10 households in Sosnowiec (in Poland) equipped with LU household monitoring systems. 44 sensors in total were installed in these 10 households. The system allows monitoring water consumption for different purposes: kitchen taps, dishwashers, toilet taps, flushing, showers and washing machines.

The data transfer status is monitored everyday as a part of the database maintenance system and additionally the status can be checked anytime using any Internet browser (Table 1).



Table 1. Status of the LU water consumption monitoring system in Sosnowiec (“Status”=1 means the

system is online, “LastUpdate” means the most recent data in the database)

In addition, 70 sensors in total have been installed in 10 households in Sosnowiec (the Apator monitoring system) and these are transmitting data in near real-time to the spatio-temporal database.

The Apator monitoring system is based on flow meters which are usually used for billing purposes. The gathered data need to be post processed before making it available for the DSS. The post processing can be considered as database verification and this is based on such rules as: deleting negative water consumptions, deleting water meter readings equal to 0, ignoring readings not consistent with the measurement temporal frequency etc.

Similarly to the LU monitoring systems, data from Apator’s installations are available and are used in decision support system.

4.1.2 Case study in Greece

At the Greek demonstration site, Skiathos, 20 households were equipped with LU monitoring systems. In total 46 sensors have been installed.

Similarly, as for the Polish case study (see Fig. 3.1), data transfer status is monitored everyday as a part of the database maintenance system and additionally the status can be checked at anytime using any Internet browser.

4.2 Urban level

At the urban level, in the two validation places (Skiathos and Sosnowiec), devices have been installed which allow 1) to control the water pressure in water distribution systems and 2) to store continuous data regarding pressure in the system. These devices are linked to the ISS-EWATUS spatio-temporal database and are used by the decision support system dedicated to water supply companies.



4.2.1 Case study in Poland

Demonstration of the decision support system on urban level in Poland is taking place in the “Kolonia Ciesle” district of the City of Sosnowiec. The Kolonia Ciesle district is a medium density residential area with 227 water users, mostly living in detached and semi-detached houses.

The Kolonia Ciesle district was chosen as a demonstration site, because: 1) it has one water source (purchasing well) and therefore allows to easy control of pressure, 2) it has considerable levels of water loses due to leakage giving an opportunity for pressure optimisation and 3) there were no obstacles to install the pressure reduction valve (PRV) and water pressure sensor in a critical point.

As mentioned above, two devices have been installed in the Kolonia Ciesle district (Figure 4):

1. Pressure reduction valve (PRV) which is installed next to the inflow of water to the water distribution system. The goal of PRV is to collect the precise information about pressure and flow rate at the inflow to the system, and secondly to enable the application of optimised pressure profiles which are products of the DSS and which should decrease leaks.

2. Pressure sensor installed in a distant point from the inlet to water distribution system (critical point – CP). The reason for installing the pressure measurement device was to obtain information about the response of pressure in the distribution system to the change of pressure in the inlet (in PRV). The information is crucial for the calibration of hydraulic models being a part of the DSS.

Both devices consist of several parts including sensors, controller, data loggers, data transmitters etc. Devices are described in detail in the Deliverable 2.1 of the ISS-EWATUS project.

Figure 4. Urban level demonstration site in Poland, Kolonia Ciesle district of Sosnowiec – Pressure

reduction valve and critical point.



Data from both devices are collected continuously and transmitted on a daily basis to the water company RPWiK in Sosnowiec and subsequently to the ISS-EWATUS spatio-temporal database. Data is being collected since February 2015. An example of pressures measured in both the PRV and the critical point are presented below (Figure 5).

Figure 5. An example of pressures measured in both the PRV and the critical point.

4.2.2 Case study in Greece

The Greek pilot location (Skiathos) was chosen to represent a different character of water distribution system and to ensure (together with the Polish site) that the DSS is efficient and useful for a wide range of geographic areas and types of water distribution systems.

The town of Skiathos is located on a small island (50 km2) and has more than 3 000 water users, which in majority are inhabitants (about 5 000) living in detached houses. Unlike Sosnowiec, Skiathos WDS is supplied by groundwater and has much older infrastructure resulting in much greater water loses. The town also has much different water use seasonality which depends on tourism and weather conditions.

Similarly to the pilot site in Poland, two types of devices were installed in Skiathos with the same goal (Figure 6):

1. Pressure reduction valve (PRV) which is installed next to the groundwater intake.

2. Pressure sensors installed in 3 distant points from the inlet to water distribution system (critical point – CP). In case of Skiathos which is much larger than the Polish demonstration site. One critical point was defined, while pressure sensors were also installed at two additional points later on, for further investigation and understanding of the WDS.



Figure 6. Urban level demonstration site in Greece, Skiathos – Pressure reduction valve and critical

points.

Data from both types of devices are collected continuously and transmitted on a daily basis to the water company DEYASK in Skiathos and subsequently to the ISS-EWATUS spatio-temporal database. An example of pressure measured in the eastern critical point in Skiathos is presented below (Figure 7).

Figure 7 Water pressure measured in one of critical points in Skiathos

30

35

40

45

50

55

60

65

70

2015-04-04 2015-05-04 2015-06-04 2015-07-04 2015-08-04 2015-09-04

Pre

ssu

re [

m]



5.Project objectives and Key Performance Indicators (KPI)

As it is mentioned before, each module of the system will be validated. As validation reflects the extent of requirement’s fulfilment and project objectives, the appropriate measures should be applied. Key performance indicators (KPI) named also key success indicators (KSI) allow to prove and illustrate that the objectives have been reached. Hence, several key performance indicators have been proposed. Below, the sets of KPIs have been distinguished respectively to the project objectives at household as well as urban level.

5.1 Household level

The documentation of ISS-EWATUS project assumes the following objectives at household level:

To increase the awareness of water consumption, an information system for gathering data about water usage is planned. The interpreted data should be presented to household consumers in an understandable way using mobile devices (tablets, smartphones).

To reduce water consumption, a household decision support system (DSS) for mobile devices should be developed. Recommendations (tips) regarding water-saving behaviour, habits will be available to the end users.

To reinforce water-saving behaviour of consumers by means of social interactions among people. A social-media platform should be developed.

5.1.1. Objective H1 – the increase of the awareness of water consumption

ISS-EWATUS workpackage

The role of workpackage in achieving objective H1

Key Performance Indicators

Description of the KPI

WP3 - Decision support system for the efficient water usage at households.

Each household is informed about water consumption in real-time mode.

KPI_H1_WP3_1 Daily consumption of water 1. Change score: awareness of own household consumption. 2. Change score: awareness of environmental impact associated with water consumption.

WP5 - Social-media platform: enabling and promoting water-saving behaviour.

Members of a households are invited to take part in the social media platform.

KPI_H1_WP5_1 Total topics created within a period of time.

KPI_H1_WP5_2 Total replies to topics within a period of time.



KPI_H1_WP5_3 Total number of water body photos and water related tips or information shared within a period of time.

KPI_H1_WP5_4 Total number of tweets disseminated on the watersocial.org website and produced from twitter users within a period of time.

WP6 - Development and simulation of adaptive water price systems.

Discovering how saving water can impact the water bill.

KPI_H1_WP6_1 Pricing scheme and water consumption

Total number of “likes” for this idea on social-media platform.

5.1.2 Objective H2 – the reduction of water consumption

ISS-EWATUS work package

The role of work package in achieving objective H2


The description of KPI measurement


Bring awareness of water consumption to the households .

KPI_H2_WP3_1 Data of consumption of water are gathered and summarize for each household before and after our intervention strategies were implemented. The difference shows the efficiency of the DSS.


The platform users will be invited to record their daily water use.

KPI_H2_WP5_1

Total new friendships made by members within a period of time

KPI_H2_WP5_2

Total new users within a period of time

KPI_H2_WP5_3

Total number of likes on topics, questions, replies and answers

KPI_H2_WP5_4 The overall water use reduction for water diary users within a period of time.


Visualisation of the effect of pricing schemes on water consumption


Data on water consumption with corresponding pricing schemes is presented



5.1.3 Objective H3 – the inducement and the reinforcement of water saving behaviour of

consumers

ISS-EWATUS workpackage

The role of workpackage in achieving objective H3




DSS is expected to influence their attitude and behaviour of the water consumption.

KPI_H3_WP3_1 Change score: water saving behaviour and attitude.


Platform users are encouraged to take part in various activities that are related water saving behaviours.

KPI_H3_WP5_1 The overall water use reduction for water diary users during water use peak time of a day.


Integration with the social media platform to create incentives to consumers.


Total users within the period of time.

5.2 Urban level

At the urban level, the first challenge of the project is to bring about the reduction of leakages at municipal level. An innovative decision support system for reducing leaks in the water delivery system by the dynamic modifications of pumping schedules has been developed.

The second objective of the project at urban level is to induce water-saving behaviour and reduce peaks in water and energy distribution loads.

5.2.1 Objective U1 – the reduction of leakages at municipal level


The role of work package in achieving objective U1



WP4 - Decision support system for efficient water management at municipal water company

The DSS at urban level includes a module which provides forecasted water demands as a driver for the pressure optimisation (based on

KPI_U1_WP4_1 Decrease in leakage. A leakage in the WDS will be calculated for two cases: for 2015 as a reference point when the pressure optimisation was not possible and for 2016 when the optimised pressure is available



hydraulic modelling). Outcomes of this module are 1) optimised pressure for the pressure reduction valves and 2) water pressure over the entire distribution system. The latter, in a combination with a power law function will be used to interpret the relation between water leaks and pressure in the project validation phase.

for both demonstration sites. The difference in leakage resulting from water pressure control before and after the development of the DSS will be used as main indicator of reduction of leakage.

Input data for the DSS include detailed information about the structure of water distribution systems. This information and particularly the network lengths can be used to identify parts of the network where leaks are the highest and where the maintenance works can be of great importance.

KPI_U1_WP4_2 Decrease of leakages per the network length. Leaks calculated as the indicator KPI_U1_WP4_1, will be used to identify sections of network most prone to leak.

In addition to the identification of sections of WDS where the leakage is dominating, the DSS provides a possibility to identify network sections with the highest peaks in pressure. In such sections the risk of leaks and bursts is higher.

KPI_U1_WP4_3 Detecting of vulnerable parts of the water distribution system Places where pressure increases during the demand peaks, will be identified at places most prone to leaks and pipe bursts.

WP4 and the DSS will be a basis for the calculation of leaks in water distribution systems.

KPI_U1_WP4_4 Economic impacts resulting from reduction of leaks. Reduction of leakage will be recalculated as the benefit estimated based on the recent water prices



WP6 - Development and simulation of adaptive water price systems

Simulation of additional incomes caused by different pricing schemes as a source for improving the pipelines maintenance.

KPI_U2_WP6_1 Additional income.

5.2.2 Objective U2 - the reduction of peaks in water and energy distribution


The role of work package in achieving objective U2



WP3 - Decision support system for the efficient water usage at households

The reduction of water consumption at households.

KPI_U2_WP3_1 Aggregation of KPI_H2_WP3_1

WP4 - Decision support system for efficient water management at municipal water company

The DSS at urban level developed as a part of the WP4 provides information regarding the water pressure in the entire distribution system. This pressure is a result of the optimisation of the PRV regulation. The optimisation has 2 goals: 1) to ensure meeting of forecasted water demands and 2) to ensure desired pressure in critical points. Consequently the DSS informs what the lowest possible pressure is at the two WDS. Application of such pressure allows to significantly decrease peaks in pressure.

KPI_U2_WP4_1 Reduction of pressure in the nights. Pressure in the WDS in the nights will be calculated for the optimised PRV operation and compared to the observed pressure in 2015 when the DSS was not applied.

KPI_U2_WP4_2 Reduction of pressure fluctuations in the critical points. Pressure in critical points will be calculated for the optimised PRV operation and compared to the observed pressure in 2015 when the DSS was not applied.

The DSS at urban level allows to calculate water leaks (indicator

KPI_U2_WP4_3 Water and energy reduction in peaks: Energy diagrams will be



KPI_U1_WP4_1). Basing on these leaks it is possible to assess what amount of water is needed to be pumped to the WDS in case of the application of optimised pressure in PRV, suggested by the DSS. Decreasing of the amount of pumped water results in the decrease of energy require for the pumping.

produced based on pumping data. For Skiathos, peak values (in both water and energy) are observed during the month of August. This indicator will quantify the reduction in average peak values during the month of August comparing system performance before and after DSS implementation. Both average peak values (for water and energy) are expected to be lower due to a reduction in leakage.

WP5 - Social-media platform: enabling and promoting water-saving behaviour

Platform users will be incentivised to reduce heavy water use in select activities.

KPI_U2_WP5_1 Reduced peaks in water and energy usage.



6.Procedure of the validation and evaluation of the ISS-EWATUS

Vast majority of actions ensuring conducting the validation of the ISS-EWATUS system will be held in the third year of the project’s duration. Achieving reliable and satisfactory results of the validation and the evaluation requires the solid engagement of system users who are presented in chapter 3. We will conduct the validation process periodically to guarantee the effective cooperation between the consortium and the system users.

Main validated components of the ISS-EWATUS are:

decision support system for the efficient water usage at households,

decision support system for the efficient water management at municipal water company,

social platform for water stakeholders,

adaptive pricing system.

In the subsections below, procedures of validation of these components are described in detail. The seasonality and data structure (gathered during the second year of the project) have been taken into account while establishing trials of the validation.

6.1 Decision support system for households

The validation process will mainly focus on the comparison of water consumption before and after the intervention strategies were implemented, therefore getting a baseline of water consumption is the starting point. Applying various intervention strategies separately or together will generate differences in water consumption. The validation will find out which intervention strategy generates most impact, and how to apply it.

The validation will be conducted in the following steps:

1. Final installations of DSS for household on tablets. (40 tablets were bought for validation purposes).

2. Meetings with representatives of households, who accept to participate in the validation.

3. Ongoing feeding the spatio-temporal database with data of water consumption during the third year of the project.

4. Extract and transform a subset of data collected in the months of the planned validation. Extract and transform the subset of data collected a year before in the same periods of time.

5. Analyse the sets of data, computing KPIs.

6. Produce the validation report of DSS at household level.

Places, participants and trials of validation of the DSS for households:

Localisation Attendees Maximum Minimum Trials of the validation

Sosnowiec (Poland)

Households/families, members of families/households

20 households (all households with ISS-EWATUS installations)

Not determinable. The number depends on the fact of how many households accept participation in thevalidation process.

1) June, July and August 2016. 2) November, December 2016 and January Skiathos Households/families, 20 households Not determinable. The



(Greece) members of families/households

(all households with ISS-EWATUS installations)

number depends on the how many households accept participation in the validation process.

2017.

Qualitative KPIs of consumer water-saving behaviour: awareness, inducement and reinforcement.

Local ISS-EWATUS partners have built up a relationship with the household members and will speak to them to publicise the online survey. Academic ethical guidelines will be followed to ensure that participants understand that, at each stage, participation is voluntary. The link to the survey will be incorporated into the external applications tab of the DSS (i.e. the tab used to access the social media site).

Identical sets of questions will be included in the original and follow up surveys. These will relate to:

1. Awareness of household’s water consumption (Objective H1) 2. Awareness of water consumption more generally (Objective H1) 3. Water consumption behaviour (Objective H2)

The choice of questions will be informed by extant research and the whole survey will be tested via a pilot study conducted with non-participants.

The sets of question will yield three scales. At the data analysis stage, these will be subject to Cronbach’s Alpha reliability test. Change scores will be derived by deducting the follow-up scale score from the original scale score. These change scores form the KPI. One-sided paired t-tests will be performed on the KPI measures to gauge the effect size and significance of the intervention. However, with a small maximum sample size, a two-stage research design and multiple outcomes, the results may be inconclusive, even where the effect size is large. In light of this limitation, at both time points, the survey will solicit qualitative (i.e. textual) responses. These may offer valuable insight into changes in perception across the course of the intervention.

6.2 Decision support system for municipal water company

The simulation module of the DSS can be relatively easy to verify through calibration procedures, i.e. comparing the observation with real-time data. However, it is still an open issue and challenge to design and adapt data-driven decision support systems and performance indicators (type and number) (Röglinger 2009). The latter should be assessed based on the objectives of the optimisation. In our case these objectives are:

Provide continuous water supply at the optimum pressure.

Reducing non-revenue water to the target level.

Forecast the water demand based on the real-time data.

The objectives are achieved by performing the following strategies:

Additional real-time monitoring system.

Pressure management via remote system and PRVs.

The critical constraints that needs to be taken into account are:

Reliable hydraulic model of networks.

Efficient database connected to the simulation model.

Reliable forecast of water demand.

Accurate spatial allocation of water demand.



Appropriate user interface enabling efficient use of the DSS.

Validation of the system performance is critical and should be focused on empirical testing whenever possible. A significant part of the validation of the DSS for municipal water companies will be based on a survey assessing the consistency with requirements and expectations of the Target Audience (water companies).

The second part of the validation will be based on the comparison of the system performance with historical data (Sojda, 2007). In such analyses an effect of the pressure management without implementation of new ICT solutions (historical monitoring data) will be compared to:

1. Outcomes of the simulation of real scenarios in order to check if the DSS can accurately reflect the modelled water distribution system.

2. Outcomes of the simulation of optimised pressure scenarios in order to check what can be achieved by the enhanced pressure management and implementation of new ICT solutions.

Pressure control management of a water distribution system is a method which is becoming more and more popular among water supply companies. Its benefits are multiple and can easily be detected by simply inspecting the structural components of the methodology. A closed district metered area (D.M.A.) supply is controlled through a pressure reducing valve (PRV). The PRV adjusts the pressure of the DMA network to the demand, as the last is revealed by monitoring of the critical point/points. This process gives out the first benefit which is the reduction of excess pressure (Ulanicki et al., 2000).

The pressure in a network is proved to be directly related to its leakage, while several empirical models are introduced to link pressure to leakage. (Sterling and Bargiela, 1984, AbdelMeguid and Ulanicki, 2010 ). Studies show that pressure has a major impact on the rate of leakage from leak openings in pipes, and that the leakage exponents in a real network can be substantially larger than the theoretical orifice exponent of 0.5. The most important reason for this behaviour is that leak areas are not fixed, but increase with increasing pressure (Cheung et al., 2010). This reveals the second benefit of the method, which is leakage reduction. In fact, leakage reduction would be the main reason why a water supply company would implement the methodology.

Furthermore, it is expected that the reduction of pressure relieves the network of all the stress onto the pipes leading to a reduction in burst incidents. The prediction of changes in frequency of bursts has been the subject of ongoing research since 1994. Pressure Management is reported (Thornton & Lambert, 2005) “to have a major influence on the other components, as reduction of excess pressures and surges usually reduce the numbers of new leaks - sometimes to a major degree – resulting in: 1) Fewer reported bursts, lower repair costs, shorter run-times, reduced repair backlogs, 2) Fewer unreported bursts, lower rates of rise of unreported leakage, less frequent interventions, lower economic volume of unreported leakage, lower annual intervention and repair costs and 3) Reduced investments in mains and services replacement programs, if criteria are based on replacement in ‘X’ number of bursts occur in ‘Y’ km of pipes in ‘Z’ years.” So, the leakage is countered in two ways, firstly, by reducing the water flowing through existing bursts in the network and secondly by reducing the bursts.

The reduction of leakage can lead to quite an appreciable saving of water resources, keeping in mind that leakage can be as high as the water demand or even more, depending on infrastructure integrity. In the case of Skiathos Island’s water network, the leakage is estimated to have occasionally risen up to 50-60% of the water demand.



Water Supply Systems are also consumers of energy, consumed in all the stages of the water production and along the entire supply chain: pumping, treatment, distribution via the network. Energy is lost due excessive supply pressure and excess supply due to water leaks or due to inefficient use of water, among others. Leakage reduction is thus naturally coupled with energy saving.

The saving of water resources has its additional impacts - social and financial. In the case of Skiathos, these impacts are straight and short-termed. Skiathos is supplied by a single drilling. The aquifer is degraded year by year due to increase of water demand through the summer tourist peaks. Incidents of complete shortage of the aquifer have already occurred, leaving the whole island without water supply during high tourism days. Moreover, the degradation of the aquifer enforces the intrusion of seawater, this leading to high salinity water in household taps, completely inappropriate for cooking or drinking. These issues, among others, reveal that through leakage reduction, a great variety of social, financial and environmental impacts is expected to improve in the everyday-life of Skiathos’ residents and tourists.

The implementation of pressure control management through a DSS-tool, that ISS-EWATUS provides, is also linked to several other useful subroutines. These subroutines add to the beneficial nature of the whole project in numerous ways. A subroutine of the DSS-tool concerns the forecast of short-term future water demand. This information, independently of the DSS-tool, can be a crucial element in avoiding future management failures. Another subroutine of the DSS-tool provides residents the ability to send an alert message to the water-company. This is expected to enhance the company’s capability to deal with bursts and leakages.

Smith (2004) sets the criteria for choosing proper indicators, by reviewing professional literature. The indicator or the information from which it is calculated should be readily available, relatively easy to understand, be about something that can be measured, something important in its own right, based on information that can be used to compare different geographical areas, be internationally comparable and there should only be a short lag time between the state of affairs referred to and the indicator becoming available.

Numerous indicators of the success of pressure control management and the ISS-EWATUS DSS-tool can be used. These indicators are related to the aforementioned impacts and can build up an easy way of evaluating the process and quantifying the impacts.

In terms of quantifying pressure reduction, one can estimate the reduction of pressure excess through the pressure diagrams of the critical points before and after implementing the DSS. This could be an outcome of the off-line testing of the control. Off-line testing has been useful for retrospective analysis of different scenarios (Ulanicki et al., 2000). The excess off pressure at critical points is a key to success. The excess ought to be minimised. It can be estimated as the integral of the area between the pressure at a critical point and the minimum customer service pressure as shown in Figure 8.



(a)

(b)

Figure 8. Critical point and PRV outlet pressure a) with basic pressure management and b) with optimised management (i2o Water Ltd, 2010)

Another indicator of pressure reduction can be a reduction of energy consumption, which can be evidenced by comparing the demand of pumping energy before and after the scheme. This is an online indicator and can be provided by the water supply company as a difference in energy cost.

The leakage reduction can be indicated through the reduction of daily pumping compared to daily pumping of previous years in corresponding timeframes such as seasons or months. Night-flow reduction can also be used as an indicator of leakage reduction, as it is widely assumed in literature that night-flow approximates the leakage (WRc-UK Water Industry, 1994).

A more long-term indicator of success would be the reduction of reported bursts in the network. This information can easily be established by the bursts-sector of the water company. Other long-term indicators, for cases such as Skiathos, may be the frequency of shortage incidents, or more indirectly the aquifer level, or the pumped water salinity. The last indicators will be more apparent through a wider timescale, such as a time period of 5 years.

These indicators are already used empirically by water-companies and depending on the water supply scheme are more or less directly related to the saving of water in the water distribution network. In the U.S Geological Survey (2002) groundwater indicators are listed as 1) groundwater level indices, with reference to hydrogeologic environment and land-use setting, 2) changes in ground water storage, regarding withdrawal, saltwater intrusion, mine dewatering and land drainage and 3) number and capacity of supply wells and artificial recharge facilities. The EPA Report on the Environment (2003) denotes trends in population served by water systems meeting all health standards as an indicator of drinking water quality. The validation process will demonstrate how all network information is transmitted and accessed by the appropriate water stakeholders, with a user-friendly interface issuing warning signs for the decrease of water pressure, etc. The validation process will also cover an appropriate interval of time to allow for the summary of results across different seasons. The expected technological impact is an improved resource efficiency of water utilities by 20% due to the use of ICT tools. The impact will be confirmed by a comparison of the before and after situation. The after situation can be considered as simulated outcomes of adaptive scenarios proposed by the ISS-EWATUS project, i.e. optimised pressure management (DSS developed in WP4) and adaptive water pricing (developed in WP6).



To validate the Decision Support System for municipal water companies and to thereby confirm that the system addresses its intended purpose, four types of indicators are proposed (Ribeiroa, 2015):

1. Management / Technical measures

2. Environmental measures

3. Economic measures

4. Social measures Places, participants and trials of validation of DSS for municipal company:

Localization Attendees Trials of the validation

Sosnowiec (Poland) Municipal utility RPWiK April 2016, July 2016, October 2016, January 2017 Skiathos (Greece) Municipal utility DEYASK

At both locations a set of key performance indicators (KPIs) listed in section Błąd! Nie można odnaleźć źródła odwołania. will be calculated on the basis of data covering at least one whole year. In addition to KPIs, a periodic survey will be carried out to assess the user acceptance and satisfaction. The assessment will be based on “Yes / No” indicators regarding User Requirements identified prior to the DSS development. Trials of the validation will be repeated quarterly and the trial frequency was chosen to enable the gathering of feedback from DSS users (RPWiK and DEYASK) and to allow to implement feedback-driven improvements to the DSS.

The Key Performance Indicators which will be used during the validation of the decision support system for the efficient water management at municipal water company level are divided into three types and are described in detail below. A) Management / Technical measures

Reduction of pressure in the nights (KPI_U2_WP4_1).

Pr =∑ (Pobsi − Psimi)

n

n

Where: Pr Average reduction in pressure at nights during the analysed season Pobs Observed pressure the in critical point Psim Simulated (optimised) pressure in the critical point i Ith day of the analysed season n Number of days in the analysed season

Reduction of pressure fluctuations in the critical points (KPI_U2_WP4_2).

PFr =∑ ((Pobs, maxi − Pobs, mini) − (Psim, maxi − Psim, mini))n

n

Where: PFr Average reduction in daily pressure fluctuation in analysed season Pobs,max Maximum observed pressure in the critical point during a day Pobs,min Minimum observed pressure in the critical point during a day Pobs,max Maximum simulated pressure in the critical point during a day



Pobs,min Minimum simulated pressure in the critical point during a day i Ith day of the analysed season n Number of days in the analysed season

Detecting vulnerable parts of the system (places where pressure increases during the demand peaks) (KPI_U1_WP4_3). Basing on the outputs from hydraulic models used for the calculation of water pressure distribution, places where the pressure increases during the demand peaks, will be identified.

Decrease in leaks (KPI_U1_WP4_1). Considering that the evaluation of the DSS will be based on the simulation of effects of pressure management rather than real pressure management in demonstration sites, the decrease in leakages will be calculated and not measured. A power law function is usually used to interpret the relation between water leaks and pressure (De Paola, Giugni, 2012; Gomes et al., 2013; Lambert, 2000):

Q = a × Pb Q leaks a discharge coefficient b pressure exponent P pressure

Leakages can be therefore calculated basing on previous hydraulic calculations (in a form of ex-post analysis) or can be included in the EPANET calculations (Morley and Tricarico, 2014; Rosmann, 2000)

Decrease of leaks per the network length (KPI_U1_WP4_2):

Qsl =QRL

LN x 365

QSL Specific water losses QRL Annual real water losses LN Network length (without service connection pipes)

“Yes / No” indicators regarding User Requirements identified prior to the DSS development. This indicator is not listed in the section 5.2 because it is not related to the achievement of main goals, i.e. reduction of leakages at municipal level and reduction of peaks in water and energy distribution loads. Required use cases are:

- To monitor the water meter consumptions - To monitor the water pressure data at a specific land-zone - To monitor the water flow - To monitor the water consumption of a previously identified land zone - To monitor the water readings from a cello device in the water network - To monitor the water pressure readings from a PRV device in the water network - To forecast the water demand for the whole water network under study - To forecast water demand for a specific land zone - To create or update or delete a specific parameter to be included in the water

demand forecasting algorithms



- To create or update or delete a specific land zone for processing (a sub area of the whole water network)

- To regulate the water pressure for the whole water network under study - To handle water pressure alerts that the system may produce. - To control and check the status of the water pressure data going to or coming from

the spatiotemporal database (migrated into a remote server). - To control and check the status of the water flow data going to or coming from the

spatiotemporal database (migrated into a remote server). - To control and check the status of the household billing data going to or coming

from the spatiotemporal database (migrated into a remote server). - To be able to load maps and/or shapefiles using a geospatial information server - To be able to load a water distribution network in a form of a shapefile using a

geospatial information server - To be able to create a new user, update the user profile or delete an existing user

of the DSS. - A water company expert may be able create, view or print reports at different

levels of processing and operations - A water company expert may be able to use complementary tools for creation of

land-zones directly on a map/shapefile. - A water company expert may be able to view and print the user manual of the

whole DSS B) Economic measures (KPI_U1_WP4_4):

Decrease in operational costs of water production as a result of reduced water leakage.

DOC = (QRL – PQRL) x CP Where:

DOC Decrease in operational costs QRL Real losses PQRL Real losses with the pressure optimisation CP Production cost

Energy savings (KPI_U2_WP4_3) by the scheduling (quantity and timing) of pumping. For the demonstration site in Skiathos (Greece) the amount of water pumped to the WSD in 2015 (before the development of the DSS) will be compared to the amount of water which includes the decrease in leaks. The reduction of energy consumed by pumps will be recalculated into reduction of costs of pumping.

Increase of the annual potential savings that can be roughly determined as:

SPA= (QUA + QAL ) x CSP + (QRL - UARLY) x (CP - CALC ) SPA Annual saving potential QUA Volume of unbilled authorised consumption QAL Apparent losses CSP Sales price QRL Real losses UARLy Unavoidable annual real losses (= UARL x 365 / 1.000) CP Production cost



CALC Costs for active leakage control measures

C) Environmental measures

Saving of surface resources (Sosnowiec)

Saving of groundwater resources by adjusting of water abstraction to predicted water demand, will be estimated based on pumping from one abstraction well in Greece;

Both indicators are equal to the decrease in leaks and decrease in water consumption.

The validation process will consist of two parts. The first one will take the form of the quantitative assessment of the system performance and will be based on the outputs from the monitoring and modelling parts of the DSS. This part will be done by the “Research Partners” of the project (IETU, CERTH with support of US and UPO – especially regarding economic indicators) and will require some input data from RPWiK and DEYASK.

The second part of the validation process will take the form of the empirical assessment of the DSS. This part will be based on a survey prepared by IETU and CERTH with a support of LU) and the Target Audience of the survey will consist of staff employed in municipal water companies (RPWiK and DEYASK).

6.3 Social-media platform

The social media platform consists of two sub-platforms, one is the website platform (http://watersocial.org), the other is the mobile app platform, available for both iPhone and Android platforms, and focused on the mobile game aspects of the whole platform.

Validation of the watersocial.org platform is a continuous process. This will ensure that the platform is running without interruption or issues.

Places, participants and trials of validation of the social media platform (watersocial.org):

Localisation Attendees Maximum Minimum Trials of the

validation

Sosnowiec

(Poland)

Individual users from

Poland. Categories

are: “unaware” and

“aware” water users

The total depends on

the number of social-

media platform users

willing to participate in

the validation process.

10 1. February –

March, 2016

2. April, May,

June 2016

3. October,

November,

December,

2016

Skiathos

(Greece)

Individual users from

Greece. Categories

are: “unaware” and

“aware” water users




willing to participate in


10

Great Britain Individual users from

the UK Categories are:

“unaware” and




10



“aware” water users willing to participate in


The social media platform is envisioned to have a comprehensive impact on the ISS-EWATUS project. More specifically it will contribute to the achievement of the following project impacts (see table below).

KPI ID Project Impacts Social Media Platform Impacts

KPI_H1_WP5 the increase of the

awareness of water

consumption

Increased social networking activities (measured in

number of visits, number of users, number of new

people joining etc.) and increased gaming activities on

promoting efferent water use.

KPI_H2_WP5 Quantifiable and

significant reduction of

water consumption

Quantifiable and significant reduction of water

consumption on a selective personal water use

activities.

KPI_H3_WP5 Peak-period reduction of

water and energy

distribution loads

Peak-period reduction of water via personal water use

activities monitoring.

KPI_U2_WP5 the reduction of peaks in

water and energy

distribution

The awareness on personal water use , discussion, and

gaming improves water use efficiency.

The key performance indicators in Table 1 are the set of quantitative measurements based on the data collected from the social media platform and its extensions including (i) mobile apps, (ii) search engine analytics and (iii) external social media platform analytics. These indicators are periodically evaluated along the pilot studies of work package 5.

Beyond the continuous validation activities there will be three pilot studies carried out with the purpose of identifying and fixing any potential technical issues. The following steps are used to conduct the 3 pilot studies, with the following objectives: (i)to ensure better quality in the data used for the indicators, (ii) to increase the pilot user sizes and (iii) to better justify the results to be observed:

1. The pilot objectives must be defined in terms of project objectives and social media platform impacts.

2. The pilot starts the user recruitment with pre-screening, in order to tag a particular user profile as “water aware” or “not water-aware”.

3. Recruited users will be given a set of instructions, following the format of a) pre-use survey b) use the platform and c) post-use survey



4. Recruited users will be followed up, and approached for the next pilot study, to detect behavioural change, if any

Additional validation will be conducted via a series of three pilot studies. These studies will take place in three countries including UK, Poland and Greece and they are scheduled for the first, second and third-fourth quarters of 2016. Each pilot study will be further based on the feedback from users.

The pilots will consist of a large involvement of water stakeholders and end users. Each country’s pilot will have a minimum of 10 participants. Participants will be asked to complete tasks on the mobile app and the WaterSocial website. The “platform use” data will be collected from members in the pilot, to test and comment on its usability and usefulness.

The user groups will consist of a mixture of both participants who are ‘water aware’ and ‘water unaware’. The evaluation procedures will comprise of two assessment measures.

The first one will be a quantitative assessment of users’ online interactions. This data will form part of the analysis of the network to observe what type of connections have been formed and by whom, and how these interactions evolve over time. Quantitative data will also be collected through users’ self-recorded water use data. The groups of water users will then be defined into above average, average, below average use categories, to mark excessive, average or limited consumption.

There are four main categories of tasks that users will be asked to complete. ● Task 1: users will record their daily water use. ● Task 2: users will share water-related photographs. ● Task 3: users will share water saving tips. ● Task 4: users will interact with other users on various social networking activities.

Tasks were chosen to provide a mixture of qualitative and quantitative data.

The second evaluation assessment will be qualitative information obtained through a pre and post study questionnaire. The first survey will establish what the users’ requirements for a social media platform are and their basic water habits. This will identify users as either ‘water aware’ or ‘water unaware’. The second survey will ask participants to detail their experience, provide feedback and suggest improvements.

Ongoing validation will also be carried out. This will involve continuous monitoring that tracks activity over time. Web analytics will be used to capture and analyse data on the use made of the WaterSocial platform and the online reach. The web analytics will provide a source of data for evidencing the impact of dissemination, public engagement and knowledge transfer activities.

6.4 Adaptive pricing model.

The possibilities of our model to distinguish between the combination of the flat rate and increasing block tariffs on the one hand and the marginal price of water on the other hand provide quite interesting conclusions: in spite of decreasing block tariffs the marginal price for majority of clients is decreasing. Such a quite surprising statement was obtained with the use of the dynamic pricing model and the database for the Skiathos case study. The details are presented in Figures 9 and 10.



Figure 9. Water tariffs in Skiathos for 2010 (Source: Data provided by DEYASK)

Figure 10. Real water prices in Skiathos (Source: Own calculations)

0

0,2

0,4

0,6

0,8

1

1,2

1,4

1,6

1,8

1

10

19

28

37

46

55

64

73

82

91

10

0

10

9

11

8

12

7

13

6

14

5

15

4

16

3

17

2

18

1

19

0

19

9

20

8

21

7

22

6

23

5

24

4

pri

ce (

ne

tt)

in €

/m3

water consuption in m3 per client per trimester (3 months)

0

2

4

6

8

10

12

14

16

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49

pri

ce (

gro

ss)

in €

/m3

water consuption in m3 per client per trimester (3 months)



Places, participants and trials of validation of the adaptive pricing model:

Localization Attendees Trials of the validation

Sosnowiec (Poland)

Municipal water company, local government

May, June, July 2016 October, November, December 2016 Skiathos (Greece) Municipal water company, local

government

Others Municipal water company, local government, water stakeholders via water social platform.

The adaptive pricing module is meant for strategic level decision makers to assess the impact of different pricing schemes. In order to validate the module, one needs to address the influence on KPIs and the pricing schemes.

Most important KPIs are: changes of water consumption due to saving water and related financial effects (change in total revenues, change in expenditure for purchasing water services)

Taking in to account the primary target of the project - reduction of water consumption and in doing so, balancing the demand with available resources - the methodology will be focused on the assessment of the water savings using the economic instrument (water tariffs). The general approach can be described as:

𝑑𝑄 = 𝑓(𝑑𝑃)

where:

dQ – changes of the quantity of a raw good or services (water supply in this case) dP = change of the price for good/service

Of course there are many other factors influencing changes in the quantity of consumed water but this module is focused on economic instruments. In more details, but still focusing only on the economic aspects, the following formula describes the responsiveness of the quantity of a raw good or service demanded to changes in its price:

𝑑𝑄 = 𝑒𝑝 ∗ 𝑑𝑝/𝑃 ∗ 𝑄

where:

ep - coefficient of price elasticity of demand P – initial price Q – quantity demanded before changes of the price, in calculation period

In the case of water demand, the term “price” is more complicated especially for mixed tariff that consist of flat rate and volumetric charges. Therefore, the real price of water is derived using the following formula:

𝑃 = 𝐸𝑥/𝑄

where:

Ex - total expenditure per client in calculation period 𝐸𝑥 = 𝐹𝑙𝑟 + 𝑉𝑐ℎ



FLr - flat rate in calculation period Vch – volumetric charge (quantity or quantities multiplied by price or prices)

The desk review gives us the estimation of the ep in local conditions, the other data was derived from water operators. Such methodology creates the possibility of predicting water saving by introducing changes in water tariffs. Such effects are related to an increase of water tariffs, however the constructed model allows to review and check the affordability aspect.

6.5 Evaluation

The word “evaluation” can have many connotations for people and specialist of various disciplines. In our case the statement that the main purpose of an evaluation can be to "determine the quality of a program by formulating a judgment" seems to be appropriate (Hurteau, Houle,Mongiat, 2009). In this case, the program is defined as the ISS-EWATUS system. While the judgment will summarise the results of the validation of all ISS-EWATUS modules: the DSS for households, the DSS for municipal water company, the social media platform and the adaptive pricing policy. Furthermore, the conclusions contained in the judgment should give an answer to the question if the planned objectives are achievable.

Hence, in accordance with DoW the purposes of the evaluation are:

to assess the quantitative reduction of water consumption,

to estimate the reduction of peaks in water and energy loads obtained by the installation of the water-saving devices installed during the course of the ISS-EWATUS project.

On the base of KPIs and the gathered data the following analysis will be performed:

• the assessment of the quantifiable and significant reduction of water consumption at households and city levels,

• the assessment of peak-period reduction of water and energy distribution loads, • the qualitative evaluation considering user acceptance and satisfaction:

a) determining what users think overall about using the ISS-EWATUS, b) estimating whether the results of ISS-EWATUS have met the expectations of water

consumers.

Conclusions

In this deliverable we have presented the plan of the validation and evaluation of all outcomes of the ISS-EWATUS project. The described plan will be realised during the third year of the project’s lifetime.

The results of the validation and evaluation will be presented in a separate document.



References Abdelmeguid, H and B. Ulanicki, Pressure and leakage management in water distribution systems via flow

modulation PRVs, WDSA 2010, the 12th annual Water Distribution Systems Analysis conference., Tucson, Arizona, USA., September 12-15, 2010.

Adrion, W. R., M. A. Branstad, and J. C. Cherniavsky, Validation, verification, and testing of computer software, ACM Computing Surveys, 14(2), 159-192, 1982.

Asian Development Bank, Data Book of Southeast Asian Water Utilities 2005. ADB, Manila, Philippines, 2007. Cheung P, G. V. Girol, N. Abe and M. Propato. Integrating water systems, Chap. In Night Flow Analysis and

Modeling for Leakage Estimation in a Water Distribution System. Taylor and Francis Group: London, UK, 2009; 509–513.

De Paola F. and Giugni M., 2012, Leakages and pressure relation: an experimental research. Drink. Water Eng. Sci., 5, 59–65, 2012, doi:10.5194/dwes-5-59-2012

D'Erchia F., Korschgen C., Nyquist M., Root R., Stine P., (2001) A Framework for Ecological Decision Support Systems: Building the Right Systems and Building the Systems Right. U.S. Geological Survey, Biological resources Division, Information and Technology Report USGS/BRD/ITR-2001-0002, 50p.

Environmental Protection Agency. 2003. Draft Report on the Environment. Washington, DC. [Online]. Available at http://www.epa.gov/indicators/roe/html/roeWaterW.htm.

Gomes R., Sousa J. and Marques A. S., 2013, The influence of pressure/leakage relationships from existing leaks in the benefits yielded by pressure management. Water Utility Journal 5: 25-32, 2013.

Hufschmidt, M.M. and Fiering, M.B. (1966) Simulation Techniques for the Design of Water Resource Systems, Harvard University Press, Cambridge, Mass.

Hurteau M. , Houle S. ,Mongiat S., (2009) How Legitimate and Justified are Judgments in Program Evaluation? SAGE Publications , DOI: 10.1177/1356389009105883, Vol 15(3): 307–319

IEEE Standard for System and Software Verification and Validation, IEEE Standard Association, IEEE, New Yourk, 2012.

i2O Water Ltd, 2010. Presentation by Andrew Burrow, presented at Sensors for Water Interest Group (SWIG) conference.

Kingdom, B., Liemberger, R. and Marin, P., The Challenge of Reducing Non-Revenue Water (NRW) in Developing Countries.World Bank, Washington, USA, 2006.

Lambert A. O., Brown T. G., Takizawa M., Weimer D., 1999, A review of performance indicators for real losses from water supply systems. Published September 1999, 48 (6) 227-237;

Lambert, A., 2000. What do we know about pressure-leakage relationships in distribution systems?. in System Approach to Leakage Control and Water Distribution Systems Management, Specialized Conference Proceedings, IWA, May, Brno (Czech Republic).

Loucks, D.P. and Van Beek, E. (2005) Water Resources systems Planning and Management: an Introduction to Methods, Models and Applications, UNESCO Press, Paris.

McKinney, D.C, Cai, X, Rosegrant, M.W., Ringler, C. and Scott C.A. (1999), Modelling Water Resources Management at the Basin Level: Review and Future Directions, International Water Management Institute, SWIM Paper 6.

Morley M.S. and Tricarico C., 2014, Pressure Driven Demand Extension for EPANET (EPANETpdd). University of Exeter, Technical Report 008/02: (revised 26/03/2014)

Rita Ribeiroa, Dália Loureiro, José Barateiro, Joanne R. Smith, Margarida Rebelo, Panagiotis Kossieris, Patricia Gerakopoulou, Christos Makropoulos, Paula Vieira, Lesley Mansfield, Framework for technical evaluation of decision support systems based on water smart metering: the iWIDGET case. Procedia Engineering 119 ( 2015 ) 1348 – 1355. 13th Computer Control for Water Industry Conference, CCWI 2015

Röglinger, Maximilian. "How to select measures for decision support systems–an optimization approach integrating informational and economic objectives." (2009).

Rosmann L.A., 2000, EPANET 2, USERS MANUAL. U.S. Environmental Protection Agency, Water Supply and Water Resources Division, National Risk Management Research Laboratory. EPA/600/R-00/057, September 2000

http://dx.doi.org/10.1061/41203%28425%29102

http://dx.doi.org/10.1061/41203%28425%29102

http://www.epa.gov/indicators/roe/html/roeWaterW.htm



Schultz, M. T., 2001, A critique of EPA’s index of watershed indicators, Journal of Environmental Management, 62, 429-442.

Smith, T. E. 2004, Water Resources Criteria and Indicators, Universities Council on Water Resources- Water Resources Update, 127: 59-67

Sojda, R.S. 2004. Empirical evaluation of decision support systems: concepts and an example for trumpeter swan management. In: Pahl--‐Wostl, C., Schmidt,S. and Jakeman, T. (eds) iEMSs 2004 International Congress: "Complexity and Integrated Resources Management". International Environmental Modelling and Software Society, Osnabrueck, Germany. http://www.iemss.org/iemss2004/proceedings/iEMSs2004_vol2.pdf

Sojda, R.S. 2007. Empirical evaluation of decision support systems: Needs, definitions, potential methods, and an example pertaining to waterfowl management. Environmental Modelling & Software 22 (2007) 269e277

Thornton, J. and Lambert A. O., Progress in practical prediction of pressure: leakage, pressure: burst frequency and pressure: consumption relationships. Proceedings of the IWA Specialised Conference ‘Leakage 2005’, Halifax, Nova Scotia, Canada, 2005.

U.S. Geological Survey. 2002. Concepts for National Assessment of Water Availability and Use. Report to Congress. Circular 1223. Reston, Virginia, 34pp. [Online].Available at http://pubs.water.usgs.gov/circ1223

Ulanicki, B., Bounds, P. L. M., Rance, J. P., & Reynolds, L., 2000. Open and closed loop pressure control for leakage reduction. Urban Water 2(2), 105-114.Walski, T. M. et al., Advanced Water Distribution Modeling And Management. Waterbury: Haestad Press, 2003.

Wardlaw R. (1999), Computer Optimisation for Better Water Allocation, Agricultural Water Management 40, 65-70.

WRc-UK Water Industry (1994). Managing leakage using night flow data. Report F. Yeh, W.W.-G. (1985) Reservoir Management and Operation Models: A State-of-the-Art-Review. Water

Resources Research 21(12): 1797-1818.

http://www.iemss.org/iemss2004/proceedings/iEMSs2004_vol2.pdf





http://pubs.water.usgs.gov/circ1223

Documents

ISS-EWATUS D7.1 PLAN OF VALIDATION AND EVALUATION · This project has received funding from the European Unions Seventh Framework Programme for research, technological development