1Physical Geography, University of Trier, 54286 Trier, Germanye-mail: [email protected], Tel.: +49 651 2014555, Fax: +49 651 2013976
Self-Organizing Maps as a tool for model time series evaluationMarcus Herbst1, Markus Casper1
Problem• Aggregating, regression based statistical performance
measures have poor discriminatory power because thecomplexity of the time series features is reduced to anumerical value.
• This implies a shortcoming for applications which depend onthe capabilities to differentiate between time series, e.g.model evaluation, optimization and uncertainty analysis.
• Motivation: Finding a technique to discriminate modelrealizations without resorting to variance based statistics.
Approach• Data: Monte-Carlo simulation with a distributed conceptual
rainfall-runoff model on the randomly sampled parameterspace is performed (N=4000 runs).
• The output time series X are normalized time step-wise to avariance of one and zero mean
• A Self-Organizing Map (SOM), consisting of nodes i=1…kwhich are characterized by a vector
with the same dimensionality as the output time series x ∈Xis trained:
(4) As k<<N, each node represents a subset of X(5) Let y be Qobserved: Find the ‘best-matching unit’ (BMU) node
c(y) with reference vector mc(y) for which
Model realizations on BMU for QobservedComparison of BMU realizations with a) manual expert calibration and b) SCE-UA optimization of RMS-Error:
Conclusions• The time series on the SOM are topologically ordered by
similarity. Simulations can be selected by different properties.• The SOM is capable of revealing information about
parameter behaviour.• It allows to constrain the ranges of sensitive parameters to a
region in the parameter space which represents e.g. Qobserved.• Models from the BMU comprise a narrow subset of similar
model alternatives that outperform the manual calibration.• Is capable to account for complexity in time series data.• Results are likely to improve by a) using larger datasets and
b) accounting for redundancies within time-series.
Further InformationHerbst, M., Casper M.C.: Towards model evaluation andidentification using Self-Organizing Maps. Hydrol. Earth Syst.Sci., 12, 657-667, 2008.This work was carried out using the SOM-Toolbox for Matlab by the “SOM Toolbox Team”, HelsinkiUniversity of Technology (http://www.cis.hut.fi/projects/somtoolbox)
Evaluation of the SOMMeans of different objective functions calculated for each node, i.e. for each subset of model realizations:
Means of parameter values for each node on the map:
Comparison with parameter scatterplots (RMSE):
[ ] nTiniii ℜ∈= µµµ ,,, 21 Km
{ }ii
yc mymy −=− min)(
BIAS
0.05
0.1
0.15
RMSE
1.3
1.4
1.5
1.6
1.7
CEFFlog
0.2
0.3
0.4
0.5
IAg
0.85
0.86
0.87
0.88
0.89
0.9
MAPE
36
38
40
42
44
46
VARmse
0.05
0.1
0.15
0.2
0.25
Rlin
0.76
0.78
0.8
0.82
= position of Qobseved on the SOM
= location of combined performance optimum
partially sensitive parameters
sensitive parameters
insensitive / interacting parameters ?
1 1.5 2 2.5 31.2
1.4
1.6
1.8
2
RM
SE
RetBasis2.5 3 3.5 4 4.5 5 5.5
1.2
1.4
1.6
1.8
2
RM
SE
RetInf 2.5 3 3.5 4 4.5 5 5.5
1.2
1.4
1.6
1.8
2
RM
SE
RetOf
2.5 3 3.5 4 4.5 5 5.51.2
1.4
1.6
1.8
2
RM
SE
S tFFRet 3 4 5 6 7 8
1.2
1.4
1.6
1.8
2
RM
SE
hL 0.2 0.4 0.6 0.8 1
1.2
1.4
1.6
1.8
2
RM
SE
maxInf
0.02 0.04 0.06 0.08 0.11.2
1.4
1.6
1.8
2
RM
SE
vL
RetInf
maxInf
RetOfRetBasis
hL
vL
StFFRet
0
5
10
15
20
25
Q [m
³/s]
Manual calibration
envelope of all simulationsobserved dischargemanual calibration
0
5
10
15
20
25
Q [m
³/s]
SCE-UA optimization
envelope of all simulationsobserved dischargeSCE-UA optimization of RMSE
14/01/95 18/02/95 25/03/95 29/04/95 03/06/95 08/07/95 12/08/95 16/09/95 21/10/950
5
10
15
20
25SOMSOMSOMSOM
Q [m
³/s]
envelope of all simulationsobserved dischargeSOM results
a
b
c
max. 38.2 max. 47.3
max. 38.2 max. 47.3
max. 38.2 max. 47.3
Recommended
