Upload
diane-kelly
View
214
Download
0
Embed Size (px)
Citation preview
ASSESSING FORECAST UNCERTAINTY FROM
SYNOPTIC TO SUB-SEASONAL SCALES.
Celeste Saulo and Juan Ruiz
CIMA (CONICET/UBA) – DCAO (FCEN –UBA)
Motivation and general context Many meteorological services run operational
ensemble prediction systems (EPS), which provide estimates of the uncertainty of the forecast.
Many of these outputs are readily available to the scientific community through, e.g. TIGGE (THORPEX Interactive Grand Global Ensemble).
Obtaining useful (valuable) information from EPS requires statistical post-processing and specific research depending on the variable/problem/region.
There is growing interest in obtaining useful information from EPS on time scales between 2 weeks and 2 months.
Motivation and general context
Active research is being pursued in numerous places on the definition of initial ensembles, multimodel (or stochastic physics) as well as on the evaluation of ensemble predictions.
During the first half of THORPEX it was realized that model error diagnosis is one area where universities and research institutions can make substantial contributions to the further development of models (and hence forecast skill), thereby supporting the relatively small community of model developers.
THORPEX = The Observing System Research and Predictability Experiment
Potential areas of research under UMI-IFAECI
Predictability studies Ensemble generation (including data
assimilation) Probabilistic forecasts Verification strategies
Related ongoing studies How sensitive are probabilistic precipitation
forecasts to the choice of calibration algorithms and the ensemble generation method? Part I: Sensitivity to calibration methods (Ruiz
and Saulo, Meteorol. Appl., 2011)Part II: sensitivity to ensemble generation
method (Ruiz, Saulo and Kalnay, Meteorol. Appl. 2011)
Three different ensemble generation strategies, using WRF regional model as the basis: Breeding (11 members)Multi-model (11 members)Pragmatic= spatially shifted ((2*m + 1)2 members, e.g.,
121)
In order to correct the effect of the ensemble systematic errors, several techniques have been developed, all of them based in the study of the relationship between error and forecasted value and in the development of statistical models to compute a calibrated probability given the forecasts of the ensemble members
A logistic regression is used to represent h(y>0|f) and a GAMMA function is used to represent h(y>tr|f,y>0)BMA → weighted + calibrated probability for each memberGAMMA-ENS →all weights are equal + calibrated probability
for each memberGAMMA→ no weights + calibration applied to the ensemble
mean WMEAN →weighted ensemble mean and then calibration is
applied
Weights associated to each member of the spatially shifted ensemble as a function of the corresponding shift in the south–north (y axis) and the west–east (x axis) directions. Negative shift values indicate southward and westward shifts respectively.
GAMMA calibration has been adopted
Continuous rankedprobability score (CRPS)
The computation of a weighted ensemble mean can lead to moderate better results; however the best choice for a weight computation algorithm is still an open question. The PQPF derived from the un-weighted ensemble mean produces, if not the best results, almost as good results as any other approach.
MM
Breeding
Shifted
24 hours forecast
Combined
48 hours forecast
Shifted combined
Shifted-Breeding
Shifted-MM
shifted multimodel 1331 membersshifted breeding 1331 members shifted combined 2541 members
The spatially shifted ensemble proves to be quite competitive at short forecast ranges,
yet its skill drops rapidly with increasing lead times
Precipitation uncertainty at these ranges is mostly related with the location of rain areas
uncertainties associated with the existence, or intensity of pp, tend to become more important with increasing lead times.
multimodel ensemble (physics) outperformed the breeding ensemble (I&BC). Still, the improvement combining both is modest
most of the PQPF limitations during summer arise from errors in model physics rather than problems in the initial and/or boundary conditions
Among the alternatives that have been evaluated, the most important improvement has been obtained with the combination of the multimodel ensemble approach (and/or the combined approach) and the spatial shift technique even at 48-hours lead time. This approach is particularly interesting and promising for implementing high resolution ensembles in small operational or research centers for which computational costs largely restrict ensemble size.
Ensemble Forecast Object Oriented Verification Method Work in progress Juan Ruiz (postdoc at LMD) and
Olivier Talagrand The method has been designed to be applied to the
500 hPa field, however it can be easily extended to other fields as well (and probably other “objects” i.e. jet streak position, low level jet maximum possition, etc).
It is based in the identification of local minima and the system associated with each local minima.
As in 500 hPa, usually low pressure systems appear in the form of troughs rather than in the form of closed systems, the geopotential height anomaly is used instead of the full 500 hPa field.
Cyclone trajectories at 500 hPa, for a particular day derived from the NCEP ensemble system
Questions for future research How much information can be obtained from the ensemble spread
about the forecast skill? Are there specific scores to quantify this relationship in terms that it becomes useful for particular applications?
Which is the most convenient way to combine different ensemble members? Is it necessary to take into account the different skill of each member? (i.e. Bayesian model averaging trying different weights against simpler techniques like logistic regression for precip)
Which kind of information/type of scores could be used to provide valuable information about weather states with more than two weeks in advance?
How can we use model error statistics to understand which processes are strongly affecting forecast quality so that key problems can be isolated and models improved?
Which methodologies should we apply to forecast probability of extreme events?