Eidgenssisches Departement des Innern EDI Bundesamt fr Meteorologie und Klimatologie MeteoSchweiz...

Eidgenössisches Departement des Innern EDIBundesamt für Meteorologie und Klimatologie MeteoSchweiz

Weather type dependant fuzzy verification of precipitation

COSMO General Meeting, Offenbach, 07.-11.09.2009

Tanja Weusthoff

2 Weather type dependant fuzzy verification | COSMO GM, 07.-11.09.2009Tanja Weusthoff

Fuzzy Verification

• „multi-scale, multi-intensity approach“

• „Fuzzy verification toolbox“ of B. Ebert

• Two methods• Upscaling (UP)• Fraction Skill Score (FSS)

• present output scale dependent

• standard setting: 3h accumulationsCOSMO-2 (2.2km): leadtimes 03-06COSMO-7 (6.6km): leadtimes 03-06,06-09,09-12,12-15

fcst obs

increasing threshold

… Upscaling (UP)

random

hitsalarmsfalsemisseshitshitshitsETS

1. Principle:

Define box around region of interest and calculate the average of observation and forecast data within this box.

3. Equitable Threat Score (ETS)

totalalarmsfalsehitsmisseshitshits random

Event if Rave ≥ thresholdNo-Event if Rave < threshold

yes no

yes Hit False Alarm

no Miss Correct negative

observation

2. Contingency Table

Q: Which fraction of observed yes - events was correctly forecast?

(Atger, 2001)

… Fraction Skill Score (FSS) (Roberts and Lean, 2005)

X XX X

X Xx XXX

1. Principle:

Define box around region of interest and determine the fraction pj and oj of grid points with rain rates above a given threshold.

3. Skill Score for Probabilities

2. Probabilities

Q: On which spatial scales does the forecast resemble the observation?

jjj op

2)(N1 FBS

FBS worst

no colocation of non-zero fractions

0 < pj < 1 = fraction of fcst grid points > threshold0 < oj < 1 = fraction of obs grid points > threshold

… Fraction Skill Score (FSS) (Roberts and Lean, 2005)

4. Useful Scales

useful scales are marked in bold in the graphics

goodbadCOSMO-7 better COSMO-2 better

COSMO-2 COSMO-7 Difference

Upscaling and Fractions Skill ScoreJun – Nov 2007

COSMO-2 vs. COSMO-7

DIFFERENCES

COSMO-7 better

COSMO-2 better

abs(median) / 0.5(q95-q05)[ 10 , Inf ]

[ 2 , 5 [

[ 5 , 10 [

[ 1 , 2 [

[ 0 , 1 [

Values = Score of COSMO-2

Size of numbers = abs(Median) / 0.5(q95-q05)measure for significance of differences

¦ COSMO-2 – COSMO-7 ¦

q(50%)

q(95%)

5% 95%

SensitivitiesOnly 00 and 12 UTC model runsCOSMO-2 & COSMO-7: leadtimes 3-6,6-9,9-12,12-15

absolute values of COSMO-2 slightly lower, but still the same pattern of differences

DIFFERENCES

COSMO-7 better

COSMO-2 better

Weather type verification: COSMO-2

4 7 5 5 7 45 31 23 15 34 7# cases

Threshold [mm/3h]

Upscaling

Fractions Skill Score

Window size: 3gp, 6.6 km

Window size: 27gp, 60 km

Window size: 3gp, 6.6 km

Window size: 27gp, 60 km

dx 0.1 0.2 0.5 1.0 2.0 5.0 10 20

NE 45 45 45 / / / / /

N 45 45 / / / / / /

NW 3 3 9 9 27 27 / /

SE 1 1 1 1 9 / / /

S 1 1 1 1 1 9 / /

SW 1 1 3 9 9 45 / /

E 9 15 27 / / / / 45

W 1 1 3 9 15 / / /

F 15 15 27 27 45 / / /

H 27 27 27 45 / / / /

L 3 9 15 27 45 / / /

dx 0.1 0.2 0.5 1.0 2.0 5.0 10 20

NE 15 / / / / / / /

N / / / / / / / /

NW 3 3 5 5 9 15 / /

SE 1 1 1 3 9 / / /

S 1 1 1 3 3 9 / /

SW 3 3 5 9 9 / / /

E 15 / / / / / / /

W 3 3 3 5 9 / / /

F 9 9 15 15 / / / /

H / 15 / / / / / /

L 3 5 9 9 15 / / /

COSMO-2, gridpoints* 2.2 km COSMO-7, gridpoints* 6.6 km

Smallest spatial scale [gridpoints] where the forecast has been useful regarding to FSS „useful scales“ definition.

# cases

% obs gridpts >= thresh (whole period)

16 14 10 7 5 2 0.5 <0.1 16 14 10 7 5 2 0.4 <0.1

% 0.1 0.2 0.5 1.0 2.0 5.0 10 20

NE 7 6 4 2 1 0.3 <0.1 <0.1

N 2 1 0.8 0.4 0.1 <0.1 <0.1 0

NW 13 11 8 5 2 0.7 0.1 <0.1

SE 17 15 12 9 6 2 0.4 <0.1

S 21 19 15 11 8 3 0.7 0.1

SW 25 22 18 13 9 3 1 0.2

E 3 2 1 0.7 0.2 <0.1 <0.1 <0.1

W 18 15 11 8 4 1 0.2 <0.1

F 17 15 11 8 5 2 0.8 0.2

H 2 2 0.8 0.5 0.2 0.1 <0.1 <0.1

L 42 36 27 19 11 3 0.8 0.1

ALL 16 14 10 7 5 2 0.5 <0.1

Fraction of observation gridpoints >= threshold climatology

Frequency of Weather Classes, June – November 2007

5 57 7

Northerly Winds (NE,N) 11 days

COSMO-2 vs.COSMO-7COSMO-2 (wc) vs.COSMO-2 (all)

COSMO-2 in northerly wind situations clearly worse than over whole period.

DIFFERENCES

COSMO-7 better

COSMO-2 better

DIFFERENCES

COSMO-2 (all) better

COSMO-2 (wc) better

Southerly Winds (SE,S) 12 days

COSMO-2 in southerly wind situations clearly better than over whole period.

DIFFERENCES

COSMO-7 better

COSMO-2 better

DIFFERENCES

COSMO-2 (wc) better

Northwesterly winds (NW) 23 days

COSMO-2 in nothwesterly wind situations at large thresholds clearly better than over whole period.

DIFFERENCES

COSMO-7 better

COSMO-2 betterD

IFFERENCES

COSMO-2 (wc) better

Flat (F) 15 days

DIFFERENCES

COSMO-7 better

COSMO-2 betterD

IFFERENCES

COSMO-2 (wc) better

COSMO-2 in flat pressure situations clearly worse than over whole period.

Summary / Conlusions

• Fuzzy verification of the D-PHASE operations period in 2007 has shown that COSMO-2 generally performed better than COSMO-7 on nearly all scales

• part of this superiority is caused by the higher update frequency, but using same model runs still shows the same pattern of differences

• the results for different weather types show large variations • best results were found for southerly winds and winds

from Northwest and West, • Northeasterly winds as well as Flat pressure situations

lead to worse perfomance of both models

UP with other scoresPOFD = FA/(CN+FA)Probability of false detection (Perfect score: 0)

FAR = FA/(H+FA)False Alarm Ratio (Perfect score: 0)

POD = H/(H+M)Probability of Detection (Perfect score: 1)

BIAS = (H+FA) / (H+M)Frequency Bias (perfect score: 1)

HK =POD – FARTrue Skill Statistik (perfect score: 1)

OR = (H*CN) / (M*FA)Odds Ratio (perfect score: infinity)

dx 0.1 0.2 0.5 1.0 2.0 5.0 10 20

COSMO-2, gridpoints* 2.2 km COSMO-7, gridpoints* 6.6 km

UP „useful scales“?

How to define usefuls scales for ETS?

Fuzzy Verifikation Intensity Scale (IS)

(Casati et al. 2004)• Transformation of Fcst and Obs into binary images on a rain/no-rain basis for the rainfall

rate thresholds. • Difference between forecast and observation = binary error image. • Decomposition into the sum of components at different spatial scales by performing a

two dimensional discrete wavelet decomposition.

Binary forecast Binary observation

Score: Mean squared error (MSE) and MSE skill score (SS) for each spatial scale component of the binary error image.

What is the relative improvement of the forecast over some reference forecast?

Intensity ScaleCOSMO-2 COSMO-7

Intensity Scale

COSMO-2

Outlook

• operational Fuzzy verification is about to start, including Upscaling and Fraction Skill Score• season, year• weather-type dependant

• Intensity scale results will further be investigated (new developments of B. Casati)

All Weatherclasses

• Overview over all weatherclasses – differences COSMO-2 minus COSMO-7 and absolute values of COSMO-2, here without bootstrapping

Northerly Winds (NE,N,NW)N NWNE

23 days4 days7 days

DIFFERENCES

COSMO-7 better

COSMO-2 better

Southerly Winds (SE,S,SW)SSE SW

7 days 45 days5 days

DIFFERENCES

COSMO-7 better

COSMO-2 better

East and West (E,W)E W

31 days5 days

DIFFERENCES

COSMO-7 better

COSMO-2 better

Flat, High, Low (F,H,L)F H L

34 days 7 days15 days

DIFFERENCES

COSMO-7 better

COSMO-2 better

Summary

JJA SON DOP

Useful scales of COSMO-2 are the same for all three time periods considered: <= 0.2 mm/3h scales larger or equal 2.2 km (= 1* gridscale ) 0.5 mm/3h scales larger or equal 6.6 km (= 3* gridscale ) 1.0 mm/3h scales larger or equal 19.8 km (= 9* gridscale ) 2.0 mm/3h scales larger or equal 33.0 km (= 15* gridscale ) 5.0 mm/3h scales larger or equal 99.0 km (= 45* gridscale )

Most prominent advantage for COSMO-2! Differences are significant.

COSMO-2 better but on many scales only slightly. Advantages on small scales/all thresholds and large thresholds/large scales. Differences are significant.

COSMO-2 better than COSMO-7 on all scales. Differences are significant.

Most frequent weatherclasses: SW (32 days) and W (21 days).

Most frequent weatherclasses: H (24 days) and NW (17 days).

Most frequent weatherclasses: SW (45 days) and H (34 days).

JJA SON DOP

NE -Relatively low values for UP-FSS: COSMO-2 better for thresholds <= 2 mm/3h, useful scales only for largest window size (99 km)

- COSMO-2 better on nearly all scales, especially for thresholds of 2-5 mm/3h. Large threshold gave no result in UP only small - scale events (averaging damps values out). Apparently, COSMO-2 could simulate those events quite well.

-Low absolute values (FSS < 0.6, UP < 0.3), no useful scales-COSMO-2 clearly better for thresholds <= 2 mm/3h (FSS)- UP: the averaged precipitation was equally well represented in both models. Only 2 mm/3h threshold better captured in COSMO-2.

N Only few days – no clear superiority of one model. Absolute values relatively low. No useful scales.

NW - COSMO-2 performed better than COSMO-7, especiall for large thresholds (> 1 mm/3h, FSS and UP).

-Many days with that weather type-Over most scales rarely different behaviour-Thresholds 5-10 mm/3h better in COSMO-2 as well as spatial scale of 6.6 km

- COSMO-2 performed better than COSMO-7, especiall for large thresholds (>= 5 mm/3h, FSS and UP).

Summary Weatherclasses I

JJA SON DOP

SE -Only few days; similar behaviour in all time periods: COSMO-2 better, especially for thresholds around 1 mm/3h and small spatial scales (FSS) and large spatial scales (UP)-Large absolute values (e.g. FSS up to 0.9, UP up to 0.66 for DOP)

S - Only few days; COSMO-2 better on most scales, most clearly for low thresholds concerning averaging (UP) and large thresholds and small spatial scales regarding probabilities (FSS), in JJA COSMO-2 clearly better also for smallest eindow size (6.6 km) for all thresholds

SW - Clearly better performance of COSMO-2, especially on smaller spatial scales

- Similar performance of both models, but COSMO-2 better for moderate thresholds (1-2 mm/3h)

- Clearly better performance of COSMO-2, especially on smaller spatial scales

Summary Weatherclasses II

JJA SON DOP

E -Only few cases (5 for DOP), COSMO-2 clearly better for low thresholds (<= 1 mm), while for a threshold of 2 mm/3h COSMO-7 better on largest spatial scales-Rather low absolute values

W - COSMO-2 clearly better on all scales, especially for thresholds <= 2 mm and smaller spatial scales

- COSMO-2 slightly better only on small spatial scales

- COSMO-2 better for thresholds <= 2 mm and especially smaller spatial scales

F -Comparable low absolute values, but COSMO-2 clearly better than COSMO-7, especially for medium and large precipitation thresholds.

-Comparable low absolute values, rarely difference in the performance of the two models. -COSMO-2 slightly better for small thresholds

- Comparable low absolute values, but COSMO-2 clearly better than COSMO-7, especially for medium and large precipitation thresholds.

H -very low absolute values (especially in SON), but COSMO-2 clearly better than COSMO-7 for precipitation thresholds <= 2 mm/3h

L - Only few cases (7 for DOP), COSMO-2 better for small spatial scales and thresholds <= 2 mm/3h

Summary Weatherclasses III

Eidgenssisches Departement des Innern EDI Bundesamt fr Meteorologie und Klimatologie MeteoSchweiz...

Documents

ENERGIE EN HET BROE~KAS-EFFEKT ’t Kan vriezen … onderzoeksterrein met inbreng vanuit bijvoorbeeld che- mie, ecologie, economie, klimatologie, meteorologie, oceanografie. Vanwege

De kalender, het jaar, de klimatologie en langetermijn ...De kalender, het jaar, de klimatologie en langetermijn-verwachtingen (deel 1) hUUg van den dool (cpc/ncep, USa) en henK de

Arbeitsbericht MeteoSchweiz Nr. 229 · in Lindenberg, Germany. During this campaign the HC2 turned out to be a very reliable sensor and the results in comparison with other humidity

Eidgenössisches Departement des Innern EDI Bundesamt für Meteorologie und Klimatologie MeteoSchweiz The new multi-layer snow model Guy de Morsier 1, Jean-Marie

Gilliam, Bryan - 'Friede Im Innern' -Strauss's Public and Private Worlds in the Mid 1930s

Eidgenössisches Departement des Innern EDI Bundesamt für Meteorologie und Klimatologie MeteoSchweiz Offene Wetter- und Klimadaten Ch. Häberli Bundesamt

Www.bmi.bund.de . Dr. Helene Groß Bundesministerium des Innern; Referat O1 Verwaltungsmodernisierung, Verwaltungsorganisation

PALM model equations Universität Hannover Institut für Meteorologie und Klimatologie, Palm-Seminar Zingst July 2004, Micha Gryschka PALM – model equations

Rights / License: Research Collection In Copyright - …41471/... · formungen im Innern des Tragwerkes. Die Ansätze dienen zur ... Dreiecke oder Vierecke). Für jedes Element wird

Vertical transports of energy and trace gases at anchor ... · disziplinäres Ökologisches Zentrum, 11 Universität Basel, Institut für Meteorologie, Klimatologie und Fernerkundung,

Horses for courses- like IPv6 profiles for german administration Constanze Bürger Bundesministerium des Innern

Bayerisches Staatsministerium des Innern, für Bau und Verkehr Disaster Control in Bavaria – Flood disaster 2013; Flood risk

Combining high-resolution numerical weather predictions ... fileEidgenössisches Departement des Innern EDI Bundesamt für Meteorologie und Klimatologie MeteoSchweiz Combining high-resolution

Klimatologie im Lichte der neueren Lehrbücher · 2020. 7. 1. · Die Klimatologie ist des weitern von der Meteorologie her um viele Forschungsergebnisse berei¬ chert worden. Neben

PA - COnnecting REpositories@iecl)erung ber Orbnung im Innern unb im galle einer ©efal)r von 9tupen; ermitß aber ... Sie übrigen 5lmtêverrid)tungen beê 9legierungê« fïattfyatterS

SÃ¼ÃŸwassereintr und Festeis in der ostsibirischen JÃ¶r ...UniversitÃ¤ Trier FB V1 - Geographie/Geowissenschaften Fach Klimatologie D-54286 Trier Die vorliegende Arbeit ist

10. Symposium Logistik Innovativ 2018 · die Oberste Baubehörde im Bayerischen Staatsministerium des Innern, für Bau und Verkehr zudem als ... Bayerisches Staatsministerium für

SYMPOSIUM ON CHALLENGES IN APPLIED HUMAN …Gehrig and Sven Kotlarski, Bundesamt für Meteorologie und Klimatologie MeteoSchweiz, Switzerland 17:00 – Fitness of heat event definition

Arbeitsbericht MeteoSchweiz Nr. 235 · CH-6605 Locarno Monti CH-1211 Genève 2 ... back until 01/09/1957 using reanalysis data of ERA40 and ERA Interim and thus provide a long time

Ozone Measurements Kenya/GAW Station - · MeteoSchweiz: Bertrand Calpini 13/05/2005 1 GAW WORKSHOP 2005 Ozone Measurements Kenya/GAW Station Bertrand Calpini, Bruno Hoegger, Gilbert