View
33
Download
6
Category
Preview:
DESCRIPTION
Weather type dependant fuzzy verification of precipitation. Tanja Weusthoff. COSMO General Meeting, Offenbach, 07.-11.09.2009. Fuzzy Verification. fcst. obs. „multi-scale, multi-intensity approach“ „Fuzzy verification toolbox“ of B. Ebert Two methods Upscaling (UP) - PowerPoint PPT Presentation
Citation preview
Eidgenössisches Departement des Innern EDIBundesamt für Meteorologie und Klimatologie MeteoSchweiz
Weather type dependant fuzzy verification of precipitation
COSMO General Meeting, Offenbach, 07.-11.09.2009
Tanja Weusthoff
2 Weather type dependant fuzzy verification | COSMO GM, 07.-11.09.2009Tanja Weusthoff
Fuzzy Verification
• „multi-scale, multi-intensity approach“
• „Fuzzy verification toolbox“ of B. Ebert
• Two methods• Upscaling (UP)• Fraction Skill Score (FSS)
• present output scale dependent
• standard setting: 3h accumulations, Jun-Nov 2007, vs Swiss radar dataCOSMO-2 (2.2km): leadtimes 03-06
COSMO-7 (6.6km): leadtimes 03-06,06-09,09-12,12-15
fcst obs
incr
easi
ng
bo
x s
ize
increasing threshold
3 Weather type dependant fuzzy verification | COSMO GM, 07.-11.09.2009Tanja Weusthoff
… Upscaling (UP)
random
random
hitsalarmsfalsemisseshits
hitshitsETS
1. Principle:
Define box around region of interest and calculate the average of observation and forecast data within this box.
3. Equitable Threat Score (ETS)
Rave
total
alarmsfalsehitsmisseshitshits random
) )((
Event if Rave ≥ thresholdNo-Event if Rave < threshold
yes no
yes HitFalse Alarm
no MissCorrect negative
observation
fore
cast
2. Contingency Table
Q: Which fraction of observed yes - events was correctly forecast?
(Atger, 2001)
4 Weather type dependant fuzzy verification | COSMO GM, 07.-11.09.2009Tanja Weusthoff
… Fraction Skill Score (FSS) (Roberts and Lean, 2005)
X XX X
X Xx XXX
x
1. Principle:
Define box around region of interest and determine the fraction pj and oj of grid points with rain rates above a given threshold.
3. Skill Score for Probabilities
2. Probabilities
Q: On which spatial scales does the forecast resemble the observation?
N
j
N
jjj op
FBS
1 1
22
N1
1FSS
N
jjj op
1
2)(N
1 FBS
FBS worst
no colocation of non-zero fractions
0 < pj < 1 = fraction of fcst grid points > threshold0 < oj < 1 = fraction of obs grid points > threshold
5 Weather type dependant fuzzy verification | COSMO GM, 07.-11.09.2009Tanja Weusthoff
… Fraction Skill Score (FSS) (Roberts and Lean, 2005)
4. Useful Scales
useful scales are marked in bold in the graphics
7 Weather type dependant fuzzy verification | COSMO GM, 07.-11.09.2009Tanja Weusthoff
goodbadCOSMO-7 better COSMO-2 better
COSMO-2 COSMO-7 Difference
- =
- =
Upscaling and Fractions Skill ScoreF
ract
ion
s sk
ill s
core
Up
scal
ing
8 Weather type dependant fuzzy verification | COSMO GM, 07.-11.09.2009Tanja Weusthoff
COSMO-2 vs. COSMO-7, bootstrapping
DIFFERENCES
COSMO-7 better
COSMO-2 better
abs(median) / 0.5(q95-q05)
[ 10 , Inf ]
[ 2 , 5 [
[ 5 , 10 [
[ 1 , 2 [
[ 0 , 1 [
Values = Score of COSMO-2
Size of numbers = abs(Median) / 0.5(q95-q05)
measure for significance of differences
¦ COSMO-2 – COSMO-7 ¦
q(50%)
q(95%)
q(5%)
5% 95%
9 Weather type dependant fuzzy verification | COSMO GM, 07.-11.09.2009Tanja Weusthoff
SensitivitiesOnly 00 and 12 UTC model runsCOSMO-2 & COSMO-7: leadtimes 3-6,6-9,9-12,12-15
absolute values of COSMO-2 slightly lower, but still the same pattern of differences
DIFFERENCES
COSMO-7 better
COSMO-2 better
10 Weather type dependant fuzzy verification | COSMO GM, 07.-11.09.2009Tanja Weusthoff
Weather type verification: COSMO-2
4 7 5 5 7 45 31 23 15 34 7# cases
Threshold [mm/3h]
Upscaling
Fractions Skill Score
Window size: 3gp, 6.6 km
Window size: 27gp, 60 km
Window size: 3gp, 6.6 km
Window size: 27gp, 60 km
dx 0.1 0.2 0.5 1.0 2.0 5.0 10 20
NE 45 45 45 / / / / /
N 45 45 / / / / / /
NW 3 3 9 9 27 27 / /
SE 1 1 1 1 9 / / /
S 1 1 1 1 1 9 / /
SW 1 1 3 9 9 45 / /
E 9 15 27 / / / / 45
W 1 1 3 9 15 / / /
F 15 15 27 27 45 / / /
H 27 27 27 45 / / / /
L 3 9 15 27 45 / / /
dx 0.1 0.2 0.5 1.0 2.0 5.0 10 20
NE 15 / / / / / / /
N / / / / / / / /
NW 3 3 5 5 9 15 / /
SE 1 1 1 3 9 / / /
S 1 1 1 3 3 9 / /
SW 3 3 5 9 9 / / /
E 15 / / / / / / /
W 3 3 3 5 9 / / /
F 9 9 15 15 / / / /
H / 15 / / / / / /
L 3 5 9 9 15 / / /
COSMO-2, gridpoints* 2.2 km COSMO-7, gridpoints* 6.6 km
Smallest spatial scale [gridpoints] where the forecast has been useful regarding to FSS „useful scales“ definition.
+
-+
-
7
4
23
5
7
45
5
31
15
34
7
# cases
% obs gridpts >= thresh (whole period)
16 14 10 7 5 2 0.5 <0.1 16 14 10 7 5 2 0.4 <0.1
13 Weather type dependant fuzzy verification | COSMO GM, 07.-11.09.2009Tanja Weusthoff
Frequency of Weather Classes, June – November 2007
47
5 57 7
45
31
23
15
34
14 Weather type dependant fuzzy verification | COSMO GM, 07.-11.09.2009Tanja Weusthoff
Northerly Winds (NE,N) 11 days
COSMO-2 vs.COSMO-7COSMO-2 (wc) vs.COSMO-2 (all)
COSMO-2 in northerly wind situations clearly worse than over whole period.
DIFFERENCES
COSMO-7 better
COSMO-2 better
DIFFERENCES
COSMO-2 (all) better
COSMO-2 (wc) better
15 Weather type dependant fuzzy verification | COSMO GM, 07.-11.09.2009Tanja Weusthoff
Southerly Winds (SE,S) 12 days
COSMO-2 vs.COSMO-7COSMO-2 (wc) vs.COSMO-2 (all)
COSMO-2 in southerly wind situations clearly better than over whole period.
DIFFERENCES
COSMO-7 better
COSMO-2 better
DIFFERENCES
COSMO-2 (all) better
COSMO-2 (wc) better
16 Weather type dependant fuzzy verification | COSMO GM, 07.-11.09.2009Tanja Weusthoff
Northwesterly winds (NW) 23 days
COSMO-2 vs.COSMO-7COSMO-2 (wc) vs.COSMO-2 (all)
COSMO-2 in nothwesterly wind situations at large thresholds clearly better than over whole period.
DIFFERENCES
COSMO-7 better
COSMO-2 betterD
IFFERENCES
COSMO-2 (all) better
COSMO-2 (wc) better
17 Weather type dependant fuzzy verification | COSMO GM, 07.-11.09.2009Tanja Weusthoff
Flat (F) 15 days
COSMO-2 vs.COSMO-7COSMO-2 (wc) vs.COSMO-2 (all)
DIFFERENCES
COSMO-7 better
COSMO-2 betterD
IFFERENCES
COSMO-2 (all) better
COSMO-2 (wc) better
COSMO-2 in flat pressure situations clearly worse than over whole period.
18 Weather type dependant fuzzy verification | COSMO GM, 07.-11.09.2009Tanja Weusthoff
Summary & Outlook
• Fuzzy verification of the 6 months period in 2007 has shown that COSMO-2 generally performed better than COSMO-7 on nearly all scales
• part of this superiority is caused by the higher update frequency, but using same model runs still shows the same pattern of differences
• the results for different weather types show large variations • best results were found for southerly winds and winds from Northwest
and West, • Northeasterly winds as well as Flat pressure situations lead to worse
perfomance of both models
• operational Fuzzy verification is about to start, including Upscaling and Fraction Skill Score
Recommended