Upload
tracy-cannon
View
213
Download
0
Embed Size (px)
Citation preview
http://www.jrc.cec.eu.int/uasa 1European Commission
DG Joint Research Centre
Formal and informal approaches to the quality of information in integrated assessment
Stefano Tarantola
January 24-25, 2002
Laxenburg, Austria
http://www.jrc.cec.eu.int/uasa 2European Commission
Information used as input to policy-making is complex, uncertain and disputed.
Established guidelines: eg NUSAP and model Pedigree schemes for the quality assurance of the decisionprocess.
Tools for Extended quality assurance
http://www.jrc.cec.eu.int/uasa 3European Commission
To set the frame
[Leamer, 1990 ] (economist):
“I propose a form of organised sensitivity analysis in
which a neighborhood of alternative assumptions is
selected and the corresponding interval of inferences
is identified. Chair in Man. Bus. Econ., UCLA
http://www.jrc.cec.eu.int/uasa 4European Commission
To set the frame
“Conclusions are judged to be sturdy only if the neighborhood of
assumptions is wide enough to be credible and the corresponding
interval of inferences is narrow enough to be useful.”
Edward E. Leamer, 1990 “Sensitivity Analysis
would help”, in Modelling Economic Series,
Edited by CWJ Granger, Clarendon Press,
Oxford. Chair in Man. Bus. Econ., UCLA
http://www.jrc.cec.eu.int/uasa 5European Commission
To set the frame
We move one step further: after characterising the interval of inferences (using e.g. the statistical variance),
Input factors should be interpreted in sensu lato: - alternative assumptions, - choice of model, - algorithmic alternatives, - poorly-known data...
Decomposition of model prediction
uncertainty
we apportion such variability to its constituents (the input factors) in the space of the assumptions (or input space).
http://www.jrc.cec.eu.int/uasa 6European Commission
The Case Study: incineration vs. landfill (Austria 1994)
No defensible choice ispossible given the uncertainties.
Robustness assessment fails:the interval of theinference is too wide
Y
http://www.jrc.cec.eu.int/uasa 7European Commission
The Case Study: incineration vs. landfill (Austria 1994)
Data16% Tu
3%STH4%
EEC5%
W_E5%
GWP9%
E/F58% Tu
Data
E/F
GWP
W_E
EEC
STH
BA
http://www.jrc.cec.eu.int/uasa 8European Commission
Settings for the sensitivity analysis
To validate or invalidate assessments
GSA used to show that the uncertainty in the decision on whether to burn or dispose solid waste depends on the choice of the system of indicators, and not on the quality of the available data.Money should not be spent to improve quality in data, but to reach a consensus on the proper system of indicators.
V(Y)=V[E(Y|Xi)]+E[V(Y|Xi)]
Tarantola et al., in Saltelli et al. Eds, (2000) Sensitivity Analysis John Wiley
http://www.jrc.cec.eu.int/uasa 9European Commission
Settings for the sensitivity analysis
Useless discussing about the use of different architectures to build the composite indicator, when these do not influence the result.
Problem simplification and dialogue optimisation
We look for those uncertain factors that have negligibleinfluence on the output.These can be fixed to the most plausible value within their range of variation.The dimensionality of the input space is then reduced.
Data16% Tu
3%STH4%
EEC5%
W_E5%
GWP9%
E/F58% Tu
Data
E/F
GWP
W_E
EEC
STH
http://www.jrc.cec.eu.int/uasa 10European Commission
Settings for the sensitivity analysis
Output uncertainty reduction
Joint use of UA and GSA (iterative procedure).
Perform UA and get a confidence interval for the output
If it is unacceptably large, acquire better knowledge on themost important factors. Perform UA again to check ...
It the output quality exceeds the requirements, the specifications on the input quality can be relaxed, starting from the less important factors. Crosetto and Tarantola (2001)
Int J Geogr Inf Science
http://www.jrc.cec.eu.int/uasa 11European Commission
Bibliography
[1] Saltelli, A., K. Chan, M. Scott, Editors, 2000, Sensitivity analysis, John Wiley & Sons publishers, Probability and Statistics series.
[2] Saltelli, A., Chan, K., Scott, M. Eds., 1999, Special Issue on sensitivity analysis, Computer Physics Communications, 117.
[3] Saltelli A., Tarantola S., and Chan K., 1999, A quantitative, model independent method for global sensitivity analysis of model output, Technometrics, 41(1), 39-56.
[4] Saltelli A., Tarantola S., Campolongo F., 2001, Sensitivity analysis as an ingredient of modelling, Statistical Science, 15(4), 377-395.