Upload
nguyennhan
View
216
Download
1
Embed Size (px)
Citation preview
Sensor data validation and reconstruction Deliverable 3.3 June 2013
2
Deliverable 3.3
Sensor data validation and reconstruction
Version 2 (last updated March 19, 2013)
List of Participants: CSIC‐UPC, UCY
List of Authors: Joseba Quevedo (UPC), Vicenç Puig (UPC), Miquel À. Cugueró (UPC), Diego García (UPC), Demetrios Eliades (UCY), Christos Panayiotou (UCY), Marios Poly‐carpou (UCY), Theofanis Lambrou (UCY)
Abstract:
In this deliverable, a methodology for data validation and reconstruction of sensor data and fault diagnosis in a water network is developed. The methodology takes into account not only spatial models but also temporal models (time series of each flowme‐ter) and internal models of the several components in the local units (pumps, valves, flows, levels, etc.). The raw data validation is inspired on the Spanish norm (AENOR‐UNE norm 500540). The methodology is based on assigning a quality level to the da‐taset considered, which is determined according to the number of data validation tests that the dataset has passed.
The methodology is applied to real‐data acquired from the Barcelona and the Limassol Water Networks, respectively. The results presented here demonstrate the ability of the proposed methodology to detect erroneous measurements coming from the sen‐sors and produce an appropriate reconstructed signal.
3
Contents 1 Introduction .............................................................................................................. 4
2 Review of existing approaches ................................................................................. 5
3 Proposed methodology for data validation and reconstruction ............................. 6
4 Application examples ............................................................................................. 15
4.1 Barcelona Water Network ............................................................................... 15
4.2 Limassol Water Network .................................................................................. 19
5 Conclusions ............................................................................................................. 21
References ...................................................................................................................... 22
4
1 Introduction In a real water network, a telecontrol system must periodically (e.g. every few minutes) acquire, store and validate data gathered by sensor measurements in order to achieve accurate monitoring of the whole network in real time.
Commonly measured variables in water network systems include hydraulic and quality parameters e.g. flow rates, nodes’ pressure, tanks’ water level, pH, conductivity, tur‐bidity, as well as disinfectant and pollutant concentrations. For each sensor measure‐ment, the data (signals) are usually represented by one‐dimensional time series.
Each sensor element measures a physical quantity and converts it into a signal that can be read by proper instrumentation. The measuring system then converts the sensor signals into values, aiming to represent a certain “real” physical quantity. These values, known as “raw data”, need to be validated before further use, in order to assure the reliability of the results obtained when using these data.
In real operation, problems affecting the communication system between the sensors set and the data logger, or the telecontrol system itself, often arise, generating missing data during certain periods of time. The data recorded by these sensors are sometimes uncorrelated and cannot be used to replace the missing data, which therefore must be replaced by a set of estimated data.
A second common problem in such systems is the lack of reliability of flowmeters (e.g. due to offset, drift and breakdowns), producing false flow data readings. These unreli‐able data must also be detected and replaced by estimated data, since flow data are used for several network water management tasks, namely: planning, investment plans, operations, maintenance and billing/consumer services and operational control (Quevedo et al., 2010a).
Furthermore, raw data may include errors such as noise, drift, outliers or due to sensor malfunctions, among others. In addition to the possible measurement deviations re‐lated to the sensor performance itself, the errors may occur due to different reasons, e.g. sensor installation problems or measurement assumption violations. Thus, it is important to provide the data system with procedures that can detect such problems and assist the user in monitoring and processing the incoming data. The data valida‐tion is an essential step to improve data reliability.
Over the last 15 years, more and more affordable on‐line sensors have become availa‐ble, leading to ever increasing acceptance of on‐line water monitoring (Tsang, 2003). These on‐line systems allow controlling mechanisms that are optimized for and re‐spond to the actual process conditions. However, this accordingly calls for data valida‐tion that is valid for the real‐time coming data. The major difference between on‐line and off‐line data validation lies in the available information and the required execution time. Generally speaking, on‐line data validation is performed based on the past time
5
series at a certain time point, e.g. without any information about future data, while off‐line data validation has the whole time series of data.
Moreover, on‐line data validation is usually required by some real time control and thus the data are used for decision support or decision making as soon as possible af‐ter being obtained.
Consequently the on‐line data validation process should be executed fast, whereas the off‐line data validation does not have such high requirement for this aspect. Another case calling for on‐line data validation is when the data read from a measuring system have very small time steps or are multiple dimensional but the practice merely re‐quires data at relatively larger time steps or in a reduced dimension form. Hence, if the raw data are not required to be kept as the evidence for later usage audition, it is not necessary to keep large amounts of data all the time due to the limited storage capaci‐ty and thus most of the data can be removed at the on‐line stage. In this sense, by on‐line data validation, the validated data represent measurements of the variables in the required form where unnecessary information from raw data has been removed.
2 Review of existing approaches
According to the nature of the available knowledge, different kinds of data validation approaches may be considered, with varying degrees of sophistication. In general, one may distinguish between elementary signal‐based (“low‐level”) methods and model‐based (“higher level”) methods (see, e.g. Denoeux et al., 1997; Mourad & Bertrand‐Krajeswski, 2002). Elementary signal based methods use simple heuristics and limited statistical information of a given sensor (Burnell, 2003; Jorgensen et al., 1998; Maul‐Kotter & Einfalt, 1998).
Typically, these methods are based on validating either signal values or signal varia‐tions. On the one hand, in the signal value‐based approach data are assessed as valid or invalid according to two different thresholds (a high one and a low one), so data is assumed to be invalid when lying outside these threshold values. On the other hand, methods based on signal variations look for strong variations (peaks in the curve) as well as lack of variations (flat curve).
Model‐based methods rely on the use of models to check the consistency of sensor data (Tsang, 2003). This consistency check is based on computing the difference be‐tween the predicted value from the model and the real value measured by the sen‐sors. Then, this difference (known as residual) is compared with a threshold value (ze‐ro in the ideal case). When the residual is bigger than the corresponding threshold, a fault is assumed in the sensor; otherwise, the sensor is assumed to work properly. Moreover, the information of all the available residuals and models allows performing fault isolation in order to discover the faulty sensor. Models are usually derived using either multivariate procedures exploiting the correlation or the analytical relations between several quantities obtained using first principles, sometimes measured at different times (“temporal redundancy”) and/or locations (“spatial redundancy”).
6
The result of data validation process may be either a binary variable indicating wheth‐er the data considered are valid or not, or a continuous validity index interpreted as a degree of confidence in these data. When the degree of confidence is too low, data can be either discarded or replaced by an estimation, computed using a statistical or physical model (see, e.g. Petit‐Renaud & Denoeux, 1998). Moreover, when using mod‐el‐based approaches for sensor data validation, model’s prediction can be also used to reconstruct the faulty sensor. Some examples of such methods in the literature applied to the water domain are:
Time‐series analysis techniques (Prescott and Ulanicki, 2001; Lobanova and
Lobanova, 2003; Bennis et al., 1997; Bennis and Kang, 2000; Crobeddu and
Bennis, 2006).
Kalman filters (Piatyszek et al. 2000; Pastres et al. 2004; Ciavatta et al. 2004).
Parity equations (Ragot and Maquin, 2006; Hamioud et al. 2005a, 2005b; Bou‐
khris et al. 2001).
Pattern recognition methods (Valentin and Denoeux 2001).
Principal Component Analysis (Nelson et al., 1996; Arteaga, 2002; Harkat et al.
2006)
3 Proposed methodology for data validation and
reconstruction
In this section, a methodology for data validation/reconstruction of sensor data and fault diagnosis in the water network is developed, taking into account not only spatial models but also temporal models (time‐series of each sensor) and internal models of the several components in the local units (pumps, valves, flows, levels, etc.). This pro‐posal allows robust isolation of unreliable sensor data which should be replaced by adequate estimated data. The methodology is mainly applied to flow and level meters, since it exploits the temporal redundancy of flow and level data. For other types of sensors requiring more complex models (pressure or water quality parameters), the reader is referred to the FP7 of the i‐Sense project (http://www.i‐sense.org/).
3.1. Data validation methodology
Raw data validation is inspired on the Spanish norm (AENOR‐UNE norm 500540). The methodology is based on assigning a quality level to the considered sensor dataset. Quality levels are assigned according to the number of tests that have been passed, as represented in Figure 1.
7
Figure 1. Raw flowmeter data validation tests
An explanation of each level is as follows:
Level 0: The communications level simply monitors whether the data are recorded
taking into account that the supervisory system is expected to collect data at a
fixed sampling time (e.g. due to problems in the sensor or in the communication
system).
Level 1: The bounds level checks whether the data are inside their physical range.
For example, the maximum values expected by the sensors are obtained by physi‐
cal limitations.
Level 2: The trend level monitors the data rate. For example, level sensor data can‐
not change more than several centimetres per minute in a real tank.
Level 3: The models level uses three parallel models:
o Local station related variables model: the local station model supervises
the possible correlation existing between the different variables in the
same local station (i.e. flow and the opening valve command in the same
pipe or pump element).
o Time series model: This model takes into account a data time series for
each variable (Blanch et al., 2009). For example, analysing historical flow
data in a pipe, a time series model can be derived and the output of the
model is used to compare and to validate the recorded data.
8
o Spatial model: The up‐downstream model checks the correlation between
historical data of sensors located in different but near local stations in the
same pipe (Quevedo et al., 2010b). For example, data of flowmeters locat‐
ed at different points of the same pipe in a transport water network allows
checking the sensor set reliability.
A decision‐tree method has been developed to invalidate data in Level 3. This method detects invalid data from the result of the three models. From that, the spatial models are very useful not only to detect problems in sensor data but also to detect leakages in pipes and to compute the balance in transport network sectors. Once invalid data is detected, the proposed method includes sensor data reconstruction process using models in Level 3 (Figure 1) in order to provide reliable sensor dataset for further tasks (e.g. maintenance, fault diagnosis).
After applying all the tests in Figure 1, if data inconsistency is detected fault isolation is performed by combining information gathered by the previous tests. For instance, if the three tests in Figure 1 detect an inconsistency in a set of two flowmeters, the sys‐tem analyses the historical data and other features of both flowmeters to diagnose the cause of the problem and to identify the sensor in faulty operation. After, all the data coming from this faulty sensor are replaced by the data of the healthy sensor installed in the same pipe.
Alternatively, the consistency between the observed and the nominal system behav‐iour may be checked, by means of a set of Physical/Temporal Parity Relations (PTPR) which relates the measured system variables under the assumption of normal (fault‐less) operation of the monitored system. An inconsistency is detected when models do not match the measurements, generating a non‐null residual. Then, the fault diagnosis mechanism is activated in order to isolate the possible fault by matching the residuals against the fault signature matrix (Puig et al., 2006). This strategy is shown in Figure .
Figure 2. Fault Detection and Isolation block diagram
A simpler strategy for data validation and reconstruction could be applied using the spatial model in the reconstruction phase but not in the validation phase, and apply
9
the other tests only based in one flowmeter data, i.e. tests 0, 1, 2, 3a and 3b for the validation phase, as it is depicted in the decision tree‐diagram for this validation strat‐egy (Figure 3), using two time series models: Holt Winters (HW) Time Series (TS) Model and Autoregressive 24 hour Time Series Model, which is detailed in the next section. In the framework of the EFFINET project, both strategies presented here will be applied and compared.
Figure 3. Block diagram of one proposed validation strategy
10
3.2. Data reconstruction methodology
The levels 0, 1, 2, 3a, 3b and 3c in Figure 3 are used to validate the raw data coming from the sensors. If any of these levels does not validate the raw data, reconstructed data is provided by the best of the three models considered for this purpose: the up‐downstream model, the Holt Winters Time Series model and the AutoRegressive 24 hour Time Series Model. The structure of these models is further explained in Section 3.3.
The best of these three models considered is used to reconstruct by the non‐validated data at time k, according to their Mean Square Error (MSE)
121
ˆ( ( ) ( ))k
i k L
MSE y i y iL
where y is the non‐validated data, y is the reconstructed data and L is the number of
previous data samples used to compute the MSE. The diagram in Figure 3 shows the proposed reconstruction procedure.
Figure 3. Block diagram of the reconstruction methodology
11
3.3. Models for data validation and reconstruction
3.3.1. Spatial model
The water network model constitutive elements and their basic relationships are in‐troduced in this section. The mass balance expression for the i‐th tank is stated as a discrete‐time difference equation
( 1) ( ) ( ) ( )i ii i in out
i
ty k y k q k q k
A
(1)
where ( )iy k is the tank level, iA is the tank surface, ( )iinq k is the manipulated
inflow and ( )ioutq k is the outflow, which may include manipulated tank outflow and
consumer demands, both given in m3/s.
Moreover, in a water network system nodes are represented as intersections of mains, which mass balance may be expressed as the static equation
( ) ( )i iin out
i i
q k q k (2)
where, similarly to Equation (1), ( )iinq k and ( )
ioutq k correspond to the inflow and out‐
flow of the i‐th subnet node, also given in m3/s.
However, to tackle real phenomena occurring in this kind of systems due to e.g. possi‐ble leakages in the pipes or bad calibration of the sensors, the mass balance in equa‐tions (1) and (2) are not strictly respected, hence a linear model correction has been proposed.
In the case of two flowmeters in the same pipe (case 1), a linear model is given by
( ) ( )in out
j l
n n
in outj 1 l 1
F t K F t M
(3)
where ( )in
j
n
inj 1
F t and ( )
out
l
n
outl 1
F t are the volumes per hour measured by the input and
output sensors, respectively ().
Figure 3. Case 1: Two flowmeters in the same pipe
Alternatively, if there is a tank between the input and the output sensor (case 2), data from the sensor level is included in the input sensor data (Figure 5).
12
Figure 4. Case 2: Two flowmeters with a tank between them
Parameters K and M in equation (3) are estimated by using real data and the least‐squares method. In the ideal case, K 1 and M 0 , respectively. Then, with the re‐siduals obtained by this model and the input sensor, using a threshold of 3σ (three times the standard deviation of the residual), outliers are found and removed. The previous steps are repeated until all the outliers are corrected, so outliers found in each step are removed from the data. At the end of this process, a linear model repre‐senting the raw data without outliers is obtained.
3.3.2. Time series models
The flow in the pipes have a daily repetitive behavior that can modelled using a TS model. TS models take advantage of the temporal redundancy of the measured varia‐bles. Thus, for each sensor with periodic behaviour, a TS model can be derived:
ˆ ( ) ( ( 1),..., ( ))ts m my k g y k y k L
where g is the TS model, for data exhibiting a periodicity of L samples.
a.‐ Holt Winter Time Series Model
A wide used method for signal forecasting is the Holt Winters (HW) triple exponential smoothing approach (Winters, 1960; Makridakis et al., 1998). This method, which is of wide use because of its simplicity and performance, may be presented in several dif‐ferent versions e.g. additive or damped trend, additive or multiplicative seasonality, single or multiple seasonality. The additive single seasonality version is considered here, which may be implemented as shown next for a forecasting horizon
ˆ ( ) ( ) ( ) ( )tsx k R k G k S k L (4)
where R is the estimate of the deseasonalized level,
( ) ( ) ( )
1 ( 1) ( 1) 0 1
R k x k S k L
R k G k
(5)
G is the estimate of the trend,
13
( ) ( ) ( 1)
1 ( 1) 0 1
G k R k R k
G k
(6)
S is the estimate of the seasonal component,
( ) ( ) ( )
1 ( ) 0 1
S k x k R k
S k L
(7)
and L is the season periodicity, , and are the HW parameters (level, trend and
season smoothing factors, respectively), x is the measured value and ˆ ( )tsx k is the TS
model forecasted value. Hence, analysing the historic records of the measured values in a certain sensor, a HW model is derived and used to validate the current acquired data by this element.
b.‐ 24 Hours Autoregressive (ARX24) Time Series Model
The aggregate daily flow model may be alternatively built on the basis of a time series modelling approach using ARIMA modelling (Box &Jenkins, 1970). A TS analysis is car‐ried out on several daily aggregate series, which consistently showed a daily seasonali‐ty, as well as the presence of deterministic periodic components (Abraham & Box,1975). A general expression for the aggregate daily time series model was derived using three main components (Quevedo, 2010a):
One‐week‐period oscillating signal with zero average value to cater for cyclic determin‐istic behaviour, implemented using a second‐order (two‐parameter) model with two oscillating modes, in s‐plane s1‐2=+/‐2π/24 j or equivalently, in z‐plane: z1‐2 = cos(2 π/24)+/‐ sin(2 π/24)j . The oscillating polynomial is presented in (8).
y( k ) 2 cos( 2 / 7 )y( k 1) y( k 2 ) (8)
An integrator that taking into account possible trends and non‐zero mean value of the flow data is presented in (9).
y( k ) y( k 1) (9)
An autoregressive component of order 21 to consider the influence of previous values within the series is presented in (10). This component plus the orders of the two com‐ponents presented in (8), (9) gathers a final order of 24 (i.e. number of samples within a day for sampling period of 1 h) for the obtained model. However, after parameter estimation and significance analysis, the models are usually reduced to a smaller num‐ber of parameters
14
y(k) = -a1y(k-1) - a2y(k-2) - a3y(k-3)- … - a21y(k-21) (10)
The three components in (8)‐(10) may be combined as follows
Δyint(k) = y(k)-y(k-1)
Δyosc(k) = Δyint(k) - 2cos(2π/24)Δyint(k-1)y(k) - Δyint(k-2)
yp(k) = -a1 Δyosc(k-1) - a2 Δyosc(k-2) - a3 Δyosc(k-3) - … - a21y(k-21)
Hence, the structure of the aggregate hourly model is presented next
yp(k) = -b1y(k-1) - b2y(k-2) - b3y(k-3) - b4y(k-4) - b5y(k-5) - b6y(k-6)
- …- b24y(k-24)
with parameters
2124
212023
21201922
2120191821
2019181720
1918171619
1817161518
1716151417
1615141316
1514131215
1413121114
1312111013
121110912
11109811
1098710
98769
87658
76547
65436
54325
43214
3213
212
11
)1)24/2cos(2(
))24/2cos(21()1)24/2cos(2(
))24/2cos(21()1)24/2cos(2(
))24/2cos(21()1)24/2cos(2(
))24/2cos(21()1)24/2cos(2(
))24/2cos(21()1)24/2cos(2(
))24/2cos(21()1)24/2cos(2(
))24/2cos(21()1)24/2cos(2(
))24/2cos(21()1)24/2cos(2(
))24/2cos(21()1)24/2cos(2(
))24/2cos(21()1)24/2cos(2(
))24/2cos(21()1)24/2cos(2(
))24/2cos(21()1)24/2cos(2(
))24/2cos(21()1)24/2cos(2(
))24/2cos(21()1)24/2cos(2(
))24/2cos(21()1)24/2cos(2(
))24/2cos(21()1)24/2cos(2(
))24/2cos(21()1)24/2cos(2(
))24/2cos(21()1)24/2cos(2(
))24/2cos(21()1)24/2cos(2(
))24/2cos(21()1)24/2cos(2(1
))24/2cos(21(
ab
aab
aaab
aaaab
aaaab
aaaab
aaaab
aaaab
aaaab
aaaab
aaaab
aaaab
aaaab
aaaab
aaaab
aaaab
aaaab
aaaab
aaaab
aaaab
aaaab
aaab
aab
ab
15
4. Application examples
The methodology described in the previous sections is illustrated using the two real water networks proposed in the EFFINET project. Three different application cases are considered, depending of the available redundancy existing due to the topological con‐figuration.
4.1. Barcelona Water Network: Preliminary Results
The strategy presented in the previous section has been already applied to the Barce‐lona Water Network. Some preliminary results showing the main features of the pro‐posed methodology is presented in this section.
The data validation and reconstruction results regarding flowmeter XX001, considering one month scenario with several faults, is depicted in Figure 6. The validation method‐ology activates several tests, such as the test of alarms (due to communications fault, in red in the validation sub‐plot), the limits of negative values of the flowmeter (in blue in the validation sub‐plot) and the incoherence of the position of both valves (closed) but the flowmeter gathers nonzero readings (in green in the validation sub‐plot) and the reconstructed signal is mainly provided by the ARX24 time series model (according to its MSE index, in green in the MSE sub‐plot).
Figure 6. Results of the validation and reconstruction of the flowmeter XX001 data
Also, the results for the validation and reconstruction of the flowmeter XX002 is de‐picted in Figure 7, considering the spatial model in Figure 8 and two time series models
16
for the reconstruction. Alarms, limits and valve inconsistence tests provide many inval‐idated data samples in the dataset, as also shown in corresponding subplot in Figure 7. In this particular case, the reconstructed data is obtained by the spatial model, be‐cause is the one with best MSE. A good coherence between both flowmeters involved in the considered spatial model is depicted in Figure 9.
Figure 7. Results of the validation and reconstruction of the flowmeter XX002 data
17
Figure 8. Spatial relationship between the flowmeters XX002 and XX003
Figure 9. Calibration and Threshold of the spatial model
The results regarding the flowmeter XX004 are shown in Figure 10, considering a spa‐tial model including two tank level sensors and four flowmeters: XX005 to XX008 (Fig‐ure 11). Almost all the invalidated data samples, detected by the limits and the valves flowmeter incoherence tests, have been reconstructed using the corresponding spatial model, as shown in Figure 12.
18
Figure 10. Validation and reconstruction results of the flowmeter XX004
Figure 11. Spatial relationship between 5 flowmeters and level sensors of a tank
19
Figure 12. Calibration and threshold of the spatial model
4.2. Limassol Water Network: Preliminary Results
Also, considering the flowmeters in Limassol water network, sensor data validation and reconstruction using methodology described in Section 3 is performed, similarly as done in Section 4.1. for the Barcelona water network. A preliminary study on this net‐work is concentrated in a part of the Mathikoloni water transport network, depicted in Figure 13.
Figure 13. Mathikoloni Water Network, part
YY002
YY001
20
In Figure 13, YY002 is the (father) flowmeter related with the output of the transport network reservoir feeding three different DMAs (Hi, Mid and Low, respectively), each one with a (son) flowmeter in its input, and with their sum values under label YY001. Hourly sampled flow measurements from April to June 2012 are considered in order to obtain the results in this section. The validation and reconstruction algorithms are im‐plemented in an on‐line fashion, which is of paramount importance for a method to be applied in a real‐time system. A simulated communication fault enduring one day (i.e. 24 samples) and a spurious fault affecting flowmeter YY002 have been used as test scenarios here. When the faults occur, the data validation process (Figure 1) notifies a fault and the data reconstruction process is activated. The available dataset is divided into an identification dataset (from sample 1 to sample 720), a validation dataset (from sample 721 to sample 1440) and a test dataset (from sample 1441 to sample 2184), where the described faults are applied. Regarding spurious fault, it is introduced at sample 2137, whilst the communication fault endures from sample 2137 to sample 2161 (24 hours). The models’ accuracy is measured by the MSE index, evaluated in the n = 48 previous values to k. The model having best MSE index when the communica‐tion fault is produced (i.e. when the data validation process is not satisfactory) is used to produce the reconstructed sensor signal.
In Figures 14 and 15, data validation and reconstruction results for the flowmeter YY002 are depicted when the simulated communication and spurious faults are affect‐ing this element, respectively. In Figure 14, the validation methodology activates the communication fault alarm (in red in the data validation sub‐plot) when the fault oc‐curs, whilst in Figure 15 several validation tests such the derivative alarm (in green in the data validation subplot) and ARX24 model alarm (in violet in the data validation subplot) are activated when the fault is produced. In both situations, data reconstruc‐tion process is activated after validation test alarms, producing the reconstructed sig‐nal (pink in the upper subplot) which is used to keep a complete record of the sensor signal.
Figure 14. YY002 data validation and reconstruction, communication fault
21
Figure 15. YY002 data validation and reconstruction, spurious fault
In both fault scenarios considered in Figures 14 and 15, the reconstructed signals are produced by the spatial model, which is the one having the best performance, meas‐ured by means of the MSE of each model (middle subplots in Figures 14 and 15) when each fault is produced.
5. Conclusions In this deliverable, a methodology has been proposed for data validation and recon‐struction of sensor datasets coming from water network applications, taking into ac‐count not only spatial models but also temporal models (time series of each flowme‐ter) and internal models of the several components in the local units (e.g. pumps, valves, flows, levels). The raw data validation is inspired on the Spanish norm (AENOR‐UNE norm 500540). The methodology consists in assigning a quality level to a certain sensor dataset, which is assigned according to the number of tests passed by the da‐taset under study.
The methodology has been successfully applied to a set of real‐data acquired from sensors installed in the Barcelona and Limassol Water Networks, respectively, in the framework of a preliminary study of these networks. The results show the ability of the proposed methodology detecting erroneous measurements and providing the corre‐sponding reconstructed signal. This methodology will be extended during the progress of the project, using the water networks under study.
22
References
Alippi C, Ntalampiras S, Roveri M. “An hmm‐based change detection method for in‐ telligent embedded sensors”. In: Neural Networks (IJCNN), The 2012 International Joint Conference on. 2012. p. 1 –7. doi:10.1109/IJCNN.2012.6252610.
Arteaga F, Ferrer A. (2002), “Dealing with missing data in MSCP: several methods, dif‐ferent interpretations, some examples”, Journal of Chemometrics, vol. 16, pp 408‐418.
Abraham B., Box G. E. P. (1975) “Linear Models, Time Series and Outliers”. University of Wisconsin‐Madison, Dept of Statistics. Tech. Report No. 438.
Bennis S., Berrada F., Kang N. (1997). “Improving single variable and multivariable techniques for estimating missing hydrological data”. Journal of Hydrology, 191(1‐4), pp. 87‐105.
Bennis, S., Kang, N. (2000). A new methodology for validating historical hydrometric data with redundant measurements. In W. R. Blain and C. A. Brebbia (Eds), Hydraulic Engineering Software VIII, WIT Press, 2000.
Blanch, J.; Puig, V.; Saludes, J.; Quevedo, J. (2009) "ARIMA Models for Data Consistency of Flowmeters in Water Distribution Networks". 7th IFAC Symposium on Fault Detec‐tion, Supervision and Safety of Technical Processes. pp. 480 – 485. Barcelona, Spain.
Boukhris A., Giuliani S., Mourot G. (2001). “Rainfall‐runoff multi‐modelling for sensor fault diagnosis”. Control Engineering Practice, Vol. 9 (6), June 2001, pp. 659‐671.
Box, G.E.P., Jenkins,G. M. (1970). Time series analysis forecasting and control. Holden‐Day.
Burnell D.(2003) “Auto‐validation of district meter data” Advances in Water Supply Management‐ Maksimovic, Butler, Memon eds., Swets & Zeitlinger Publishers, The Netherlands.
Ciavatta, S., Pastres, R., Lin, Z., Beck, M.B., Badetti, C., Ferrari, G. (2004). “Fault detec‐tion in a real‐time monitoring network for water quality in the lagoon of Venice (Ita‐ly)”. Water Science and Technology, Vol. 50, No 11, pages 51‐58, 2004.
Crobeddu, E., Bennis, S. (2006). “Data acquisition, validation and forecasting for a combined sewer network”. In V. Popov, A.G. Kungolos, C.A. Brebbia and H. Itoh (Eds), Waste Management and the Environment III, WIT Press.
Denoeux, T., Boudaoud, N., Canu, S., Dang, V.M., Govaert, G., Masson, M., Petitrenaud, S., Soltani, S. (1997). “High level data fusion methods”. Technical Report CNRS/EM2S/330/11‐97v1.0, Université de Technologie de Compiègne, Compiègne, France, November 1997.
Hamioud, F., Joannis, C., Ragot, J. (2005a). Fault diagnosis for validation of hydrometric data collected from sewer networks. 10th International Conference on Urban Drain‐age, 10ICUD, Copenhagen, Denmark, August 21‐26, 2005.
23
Hamioud, F., Joannis, C., Ragot, J. (2005b). Localisation de défauts de capteur pour la validation des mesures hydrométriques issues de réseaux d'assainissement. 20ème colloque sur le traitement du signal et de l'image GRETSI 2005 ‐‐ Louvain la Neuve Bel‐gique, 6‐9 septembre 2005.
Harkat, M.F., Mourot, G., Ragot, J. “An improved PCA scheme for sensor FDI: Applica‐tion to an air quality monitoring network”. Journal of Process Control, Vol. 16, Issue 6, July 2006, Pages 625‐634.
Jörgensen H.K, Rosenörn S., Madsen H., Mikkelsen P.S. (1998) “Quality control of rain data used for urban run‐off systems”. Water Science and Technology, 37(11), pp 113‐120.
Lobanova H.V. , Lobanova G. V. (2003). “ Approach for Restoration of Missing Data, Long‐term Time Series and Generalization of Results” in ” Advances in Water Supply Management‐ Maksimovic, Butler, Memon eds., Swets & Zeitlinger Publishers, The Netherlands.
Maul‐Kötter, B., Einfalt T. (1998). “Correction and preparation of continuously meas‐ured rain gauge data: a standard method in North Rhine‐Westphalia”. Water Science and Technology, 37(11), pp 155‐162.
Maidment, D. R., E. Parzen (1984a). “Time patterns of water uses in six Texas cities”. Journal of Water Resources Planning and Management, ASCE, 110(1), 90‐106.
Maidment, D.R., E. Parzen (1984b). “Cascade model of monthly municipal water use”. Water Resources Research, 20(1), 15‐23.
Maidment, D.R., Miaou, S.P., M.M. Crawford (1985). “Transfer function models of daily urban water use” Water Resources Research, 21 (4), 425‐432.
Makridakis S, Wheelwright S, Hyndman R. Forecasting methods and applications. John Wiley & Sons, 1998.
Matheson, D., Jing, C., Monforte, F. (2004). “Meter Data Management for the Electrici‐ty Market”. 8th International Conference on Probabilistic Methods Applied to Power Systems, Iowa State University, Ames, Iowa, September.
Mourad, M., Bertrand‐Krajeswski, J.L. (2002). “A method for automatic validation of long time series of data in urban hydrology”. Water Science and Technology Vol. 45, No 4‐5, pages 263‐270, 2002.
Nelson P., Taylor P., MacGregor J. (1996), “Missing data methods in PCA and PLS: Score calculations with incomplete observations”, Journal of Chemometrics and Intelligent Laboratory Systems, vol. 35, pp 45‐65.
Pascual J, Romera J, Puig V, Creus R, Minoves M. “Operational predictive optimal con‐ trol of barcelona water transport network”. In: World Congress of the International Federation of Automatic Control. Proceedings of the 18th IFAC World Congress. Milan; 2011. .
24
Pastres, R., Ciavatta, S., Solidoro, C. (2003). “The Extended Kalman Filter (EKF) as a tool for the assimilation of high frequency water quality data”. Ecological Modelling, Vol. 170, Issues 2‐3, 15, Pages 227‐235, 2003.
Petit‐Renaud, S., Denoeux, T. “A neuro‐fuzzy system for missing data reconstruction”. 1998 IEEE Workshop on Emerging Technologies, Intelligent Measurement and Virtual Systems for Instrumentation and Measurement, Saint‐Paul, USA, May 1998.
Piatyszek, E., Voignier, P., Graillot, D. (2000). “Fault detection on a sewer network by a combination of a Kalman filter and a binary sequential probability ratio test“. Journal of Hydrology, Volume 230, Issues 3‐4, Pages 258‐268, 2000.
Prescott S.L., Ulanicki B. (2001) “Time Series Analysis of Leakage in Water Distribution Networks” in Water Software Systems Theory and Applications. Research Studies Press, England.
V. Puig, A. Stancu, T. Escobet, F. Nejjari, J. Quevedo y R. Patton, «Passive robust fault detection using interval observers: Application to the damadics benchmark problem,» Control Engineering Practice, vol. 14, nº 6, pp. 621‐633, 2006.
Quevedo, J.; Pascual, J.; Puig, V.; Saludes, J.; Espin, S.; Roquet, J. (2012) "Data valida‐tion and reconstruction of flowmeters to provide the annual efficiency of ATLL transport water network in Catalonia". Proceedings of IWC. New Developments in IT & Water. Amsterdam, Nederlands.
Quevedo, J., Puig, V., Cembrano, G., Blanch. J. (2010a): “Validation and reconstruction of flowmeter data in the Barcelona water distribution network”. Control Engineering Practice Journal, 18 (6), pp. 640‐651.
Quevedo, J.; Blanch, J.; Puig, V.; Saludes, J.; Espin, S.; Roquet, J. (2010b): “Methodology of a data validation and reconstruction tool to improve the reliability of the water network supervision”, Water Loss Conference 2010, Sao Paulo, Brazil.
Ragot, J., Maquin, D. (2006). “Fault measurement detection in an urban water supply network”. Journal of Process Control, Volume 16, Issue 9, Pages 887‐902, 2006.
Tsang, K.M. (2003). “Sensor data validation using gray models”. ISA Transactions 42, 9–17.
Valentin, N., Denoeux, T. (2001) “A neural network‐based software sensor for coagula‐tion control in a water treatment plant”. Intelligent Data Analysis, 5:23‐39.
Winters PR. “Forecasting sales by exponentially weighted moving averages”. Manage‐ ment Science 1960; 6(52):324–42.