5.5.2003 1

The development of statistical interpretation and adaptation of NWP at FMI

Juha Kilpinen, Ahti Sarvi and Mikael JokimäkiFinnish Meteorological Institute

http://www.fmi.fi/

• Past operational methods: – Perfect Prognosis (with multiple regression)– Kalman filtering– Decision threes

• Present pre-operational methods:– Fuzzy systems for points– Perfect Prognosis for grid points

5.5.2003 2

The development of statistical interpretation and adaptation of NWP at FMI

• Past operational methods: – Perfect Prognosis (with multiple regression)

• for three stations, several parameters– Kalman filtering

• temperature, min/max temperatute, off shore winds, PoP tests, for stations

– Decision threes• several parameters, for grid data

5.5.2003 3

The development of statistical interpretation and adaptation of NWP at FMI

• Present pre-operational methods:– Fuzzy systems for points

• ECMWF temperature– Perfect Prognosis for grid data

• temperature/ground temperature/Min-Max • HIRLAM and ECMWF data• to be used within the grid editing process

5.5.2003 5

Observations(Global & Local)

Customers:Public WebBusiness:

MediaAviationIndustrySecurity:

General publicAuthorities

Editing by forecasters (FMI)Post processing

Production Servers

(FMI)

Real time Database

(FMI)Post processing

(e.g. statistical adaptation)

HIRLAM model (CSC)

Climate database (FMI)

Forecasts

ObservationsBoundariesForecasts

Graphicstext forecasts

etc.

ECMWF

MonitoringSMS(FMI)

Forecasters: Manual

products

Forecasting process at FMI

5.5.2003 6

The Grid EditorSmart Tools: ability to makeScripts to perform more Complicated and oftenRepeated editing actions inA more easy manner (suitableAlso for adaptation purposes)

IF (N>5) T=T+3

5.5.2003 7

MAE of temperature forecasts (3 stations, 9 seasons, 0.5-5 days)

0

0,5

1

1,5

2

2,5

3

3,5

4

4,5

Dec 2000-Feb 2001

Mar 2001-May 2001

Jun 2001-Aug 2001

Sep 2001-Nov 2001

Dec 2001-Feb 2002

Mar 2002-May 2002

Jun 2002-Aug 2002

Sep 2002-Nov 2002

Dec 2002-Feb 2003

Mean

MA

E(C

)

Forecaster_Points

Forecaster_Editor

Models

Best_model

Centralized editing on commercial side

5.5.2003 8

HIRLAM DMO and Obs (25.4.2001)

HIRLAM PPM and Obs (25.4.2001)

Error ~ 5-9 degreesmax error 10 degrees

Error ~ 10-15 degreesmax error -20 degrees

5.5.2003 9

Perfect Prognosis method for temperature forecastingJuha Kilpinen

2100 grid points, HIRLAM and ECMWF models•applies same models for both data sources (HIRLAM/ECMWF)•developmental data from TEMP’s of Jokioinen (02935) and Sodankylä (02836), 20 years of data •separate models for 00UTC, 03UTC, 06UTC, 09UTC,12UTC, 15UTC, 18UTC and 21 UTC (see Fig.)•over sea or lakes DMO is used•data stratification for four seasons, overlap of seasons 1 month (see Fig.)•TEMP data from surface up to 500 HPa used, also derived new predictors used•multiple linear regression (Systat 10)•forward selection of predictors, a new predictor should increase the reduction of variance of the model by at least 0.5%.

5.5.2003 10

Derived predictors for PPM

• FF850 = SQRT(ABS(V850*U850))• TYPE_PRHFF = (P_P0H-949)/15.6+(24-FF850)/3.93+(100-

RH850)/28• CL_MAX =

MAX(RH500*RH500/100,RH700*RH700/100,RH850*RH850/100)

• TYPE_PCLFF = (P_P0H-949)/15.6+(100-CL_MAX)/28.2+(24-FF850)/3.93

• P_P0H2 = P_P0H-1013• Z8502 = Z850-1500• Z7002 = Z700-3000• Z5002 = Z500-5200• COSINUS = COS(2*3.1417*JUL/360)• SINUS = SIN(2*3.1417*JUL/360)

5.5.2003 11

0

0,5

1

1,5

2

2,5

3

3,5

4

18 21 0 3 6 9 12 15 18 21 0

Jokioinen 00 UTC

Jokioinen 12 UTC

Jokioinen 00 UTC

Estimation error of dependent PPM model

UTC

5.5.2003 12

0

1

2

3

4

5

6

18 21 0 3 6 9 12 15 18 21 0

Sodankylä 00 UTC

Sodankylä 12 UTC

Sodankylä 00 UTC

Estimation error of dependent PPM model

UTC

5.5.2003 13

12 UTC TEMP

00 UTC TEMP

SYNOP 09 12 15 18 21 00 03 06 UTC

Connections of TEMP and SYNOP data in estimation

5.5.2003 14

Winter (5 months)

Spring (3 months)

Summer (5 months)

Autumn (3 months)

Winter (5 months)

Data stratification and overlap of seasonal models

overlap 1 month

5.5.2003 15

Perfect Prognosis for temperature forecasts: A typical model Data for the following results were selected according to:

(SEASON_SF= 2) AND (HH= 12)4 case(s) deleted due to missing data. Dep Var: T2M_P0H N: 548 Multiple R: 0.98533425 Squared multiple R: 0.97088359 Adjusted squared multiple R: 0.97050616 Standard error of estimate: 1.17789326 Effect Coefficient Std Error Std Coef Tolerance t P(2 Tail) CONSTANT -25.23580650 22.51961688 0.00000000 . -1.12061 0.26295Z500 0.00951429 0.00397318 0.21740651 0.0065415 2.39463 0.01698T700 -0.14111429 0.05617324 -0.12111370 0.0231975 -2.51213 0.01229RH700 -0.01421594 0.00192526 -0.06048159 0.8036540 -7.38390 0.00000T850 -0.49766238 0.03697928 -0.43038524 0.0527208 -1.34E01 0.00000COSINUS -1.75269612 0.19284168 -0.10759268 0.3847601 -9.08878 0.00000Z8502 0.25777160 0.00698847 3.60146923 0.0056557 36.88525 0.00000P_P0H2 -2.08679352 0.04590369 -3.40166499 0.0096300 -4.54E01 0.00000  Analysis of VarianceSource Sum-of-Squares df Mean-Square F-ratio P Regression 2.49825E+04 7 3.56892E+03 2.57232E+03 0.00000000Residual 7.49214E+02 540 1.38743253  ------------------------------------------------------------------------------- Durbin-Watson D Statistic 1.52355145First Order Autocorrelation 0.23719694

5.5.2003 16

Perfect Prognosis for temperature forecasts: A typical model

Data for the following results were selected according to: (SEASON_WS= 2) AND (HH= 00)4 case(s) deleted due to missing data.

Dep Var: T2M_P0H N: 3335 Multiple R: 0.949 Squared multiple R: 0.901 Adjusted squared multiple R: 0.901 Standard error of estimate: 1.404 Effect Coefficient Std Error Std Coef Tolerance t P(2 Tail) CONSTANT 22.426 0.529 0.000 . 42.386 0.000T850 0.063 0.023 0.065 0.054 2.753 0.006COSINUS -2.238 0.129 -0.147 0.418 -17.372 0.000Z8502 0.134 0.004 2.186 0.006 30.671 0.000P_P0H2 -1.050 0.036 -1.966 0.007 -29.113 0.000RH850 0.022 0.002 0.104 0.516 13.627 0.000SINUS -1.114 0.058 -0.155 0.460 -19.281 0.000TYPE_PNFFP0H -0.916 0.021 -0.365 0.433 -44.007 0.000  Analysis of VarianceSource Sum-of-Squares df Mean-Square F-ratio P Regression 59581.994 7 8511.713 4320.200 0.000Residual 6554.898 3327 1.970  -------------------------------------------------------------------------------*** WARNING ***Case 11951 has large leverage (Leverage = 0.012) Durbin-Watson D Statistic 1.670First Order Autocorrelation 0.165

5.5.2003 17

Perfect Prognosis for temperature forecasts:

•The models of Jokioinen (02935) used south of Jokioinen, the models of Sodankylä (02836) used north of Sodankylä and interpolation between these stations•PPM calculated after every HIRLAM run (4 times a day) and for ECMWF data once a day to a grid•Verification results available for stations (ME, MAE,...)•Verification results available for grid (based on MESAN analysis)•Timeseries of forecasts and observations for stations

5.5.2003 18

Location of Jokioinen and Sodankylä; ECWMF PPM and DMO

5.5.2003 20

Verification results of PPM: Mean Error ME (bias) Mean Absolute Error MAE

HIRLAM 00 UTC analysis

ECMWF 12 UTC corresponding to the same valid time

+06h

+48h

18h

5.5.2003 21

5.5.2003 22

5.5.2003 23ECMWF MAE Summer 2002

5.5.2003 24

-2,5

-2

-1,5

-1

-0,5

0

0,5

1

1,5

2

2,5

6 9 12 15 18 21 24 27 30 33 36 39 42 45 48

ME

MEppm

MAE

MAEppm

Error of HIRLAM (and PPM) temperature forecasts (summer 2002 30 stations)

Forecast length (hours)

5.5.2003 25

-3

-2

-1

0

1

2

3

6 9 12 15 18 21 24 27 30 33 36 39 42 45 48

ME

MEppm

MAE

MAEppm

Error of ECMWF (and PPM) temperature forecasts (summer 2002 30 stations)

Forecast length (hours)

5.5.2003 26

Dep Var: T2M_09UTC N: 3290 Multiple R: 0.962 Squared multiple R: 0.926 Adjusted squared multiple R: 0.926 Standard error of estimate: 1.354 Effect Coefficient Std Error Std Coef Tolerance t P(2 Tail)CONSTANT 25.947 0.238 0.000 . 109.076 0.000V700 -0.007 0.004 -0.011 0.766 -2.042 0.041T850 -0.411 0.017 -0.377 0.089 -23.784 0.000COSINUS -1.556 0.092 -0.091 0.771 -16.947 0.000CL_MAX -0.085 0.015 -0.031 0.803 -5.861 0.000Z8502 0.246 0.003 3.617 0.010 77.126 0.000P_P0H2 -1.995 0.027 -3.249 0.012 -73.829 0.000

Dep Var: T2M_12UTC N: 3289 Multiple R: 0.983 Squared multiple R: 0.967

Adjusted squared multiple R: 0.967 Standard error of estimate: 0.945

Effect Coefficient Std Error Std Coef Tolerance t P(2 Tail)CONSTANT 30.611 0.166 0.000 . 184.289 0.000V700 -0.026 0.002 -0.039 0.767 -10.829 0.000T850 -0.506 0.012 -0.445 0.089 -41.932 0.000COSINUS -0.753 0.064 -0.042 0.771 -11.746 0.000CL_MAX -0.317 0.010 -0.110 0.803 -31.163 0.000Z8502 0.274 0.002 3.864 0.010 123.162 0.000P_P0H2 -2.218 0.019 -3.459 0.012 -117.535 0.000

Jokioinen Summer 12 UTC TEMP PPM Models for 09 UTC and 12 UTC

5.5.2003 27

-1

-0,5

0

0,5

1

1,5

2

2,5

3

3,5

6 9 12 15 18 21 24 27 30 33 36 39 42 45 48

ME

MEppm

MAE

MAEppm

Error of HIRLAM (and PPM) temperature forecasts (autumn 2002 30 stations)

Forecast length (hours)

5.5.2003 28

Temperature error of HIRLAM (and PPM) at Jokioinen (02935) summer 2002

-2,5

-2

-1,5

-1

-0,5

0

0,5

1

1,5

2

2,5

3

6 9 12 15 18 21 24 27 30 33 36 39 42 45 48

ME

MEppm

MAE

MAEppm

Forecast length (hours)

5.5.2003 29

Temperature error of ECMWF (and PPM) at Jokioinen (02935) summer 2002

-3

-2

-1

0

1

2

3

6 9 12 15 18 21 24 27 30 33 36 39 42 45 48

ME

MEppm

MAE

MAEppm

Forecast length (hours)

5.5.2003 30

Temperature error of HIRLAM (and PPM) at Sodankylä (02836) summer 2002

-2

-1,5

-1

-0,5

0

0,5

1

1,5

2

2,5

3

6 9 12 15 18 21 24 27 30 33 36 39 42 45 48

ME

MEppm

MAE

MAEppm

Forecast length (hours)

5.5.2003 31

Temperature error of ECMWF (and PPM) at Sodankylä (02836) summer 2002

-3

-2

-1

0

1

2

3

6 9 12 15 18 21 24 27 30 33 36 39 42 45 48

ME

MEppm

MAE

MAEppm

Forecast length (hours)

5.5.2003 32

-0,5

0

0,5

1

1,5

2

2,5

3

3,5

6 9 12 15 18 21 24 27 30 33 36 39 42 45 48

ME

MEppm

MAE

MAEppm

Error of ECMWF (and PPM) temperature forecasts (autumn 2002 30 stations)

Forecast length (hours)

5.5.2003 33

-4

-3

-2

-1

0

1

2

3

4

5

6 9 12 15 18 21 24 27 30 33 36 39 42 45 48

ME

MEppm

MAE

MAEppm

Error of HIRLAM (and PPM) temperature forecasts (spring 2002 30 stations)

Forecast length (hours)

5.5.2003 34

-3

-2

-1

0

1

2

3

6 9 12 15 18 21 24 27 30 33 36 39 42 45 48

ME

MEppm

MAE

MAEppm

Error of ECMWF (and PPM) temperature forecasts (spring 2002 30 stations)

Forecast length (hours)

5.5.2003 35

-2

-1

0

1

2

3

4

5

6 9 12 15 18 21 24 27 30 33 36 39 42 45 48

ME

MEppm

MAE

MAEppm

Error of HIRLAM (and PPM) temperature forecasts (winter 2002-2003 30 stations)

Forecast length (hours)

5.5.2003 36

0

0,5

1

1,5

2

2,5

3

3,5

4

6 9 12 15 18 21 24 27 30 33 36 39 42 45 48

ME

MEppm

MAE

MAEppm

Error of ECMWF (and PPM) temperature forecasts (winter 2002-2003 30 stations)

Forecast length (hours)

5.5.2003 37

y = -0,0005x3 - 0,0284x2 - 0,3902x - 0,7765R2 = 0,074

y = -2E-08x6 - 1E-06x5 - 1E-05x4 + 5E-05x3 - 0,0183x2 - 0,3713x - 0,9014R2 = 0,0795

-15

-10

-5

0

5

10

15

-50 -40 -30 -20 -10 0 10

Residuals versus Estimates Sodankylä PPM model in Winter (00 UTC)

5.5.2003 38

-3

-2

-1

0

1

2

3

4

6 9 12 15 18 21 24 27 30 33 36 39 42 45 48 51 54 57 60 66 72 78 84 90 96 102 108 114 120 126

ME

MEppm

MAE

MAEppm

Error of ECMWF (and PPM) temperature forecasts (summer 2002 30 stations)

Forecast length (hours)

5.5.2003 39

0

1

2

3

4

5

6

6 9 12 15 18 21 24 27 30 33 36 39 42 45 48 51 54 57 60 66 72 78 84 90 96 102 108 114 120 126

ME

MEppm

MAE

MAEppm

Error of ECMWF (and PPM) temperature forecasts (winter 2002-2003 30 stations)

Forecast length (hours)

5.5.2003 40

-1,5

-1

-0,5

0

0,5

1

1,5

2

2,5

3

3,5

6 9 12 15 18 21 24 27 30 33 36 39 42 45 48

HME

HMEppm

HMAE

HMAEppm

EME

EMEppm

EMAE

EMAEppm

Error of HIRLAM and ECMWF (& PPM) temperature forecasts in Finland (one year, 30 stations)

Forecast length (hours)

5.5.2003 41

A Fuzzy system for adaptation of ECMWF T2m forecasts

Ahti Sarvi

• Fuzzy system has been applied to correct the temperature (T2m) forecasts of ECMWF. These forecasts as well as HIRLAM forecasts have errors (systematic) typically in stable conditions (inversions). The objective of fuzzy system approach has been to utilize the information included in forecasts and corresponding observations by constructing a set of 2m temperature estimators based on the verifications of the most recent 27 successive 10 day forecasts.

• The set of estimates given by these estimators may involve missing values and outliers, but in fuzzy set approach these contradictions in the data do not cause problems provided that the amount of the information included in the set of estimates input to the system is sufficient.

5.5.2003 42

A Fuzzy system for adaptation of ECMWF T2m forecasts

• In an iterative solution process of fuzzy system a membership function, the values of which are normalized between zero and one, assigns the grade of membership for each estimate and zero for messy data and thus excludes the messy data from the solution and prevents it from corrupting the final estimate given by the system.

• The verification results for a short test period are presented

5.5.2003 43

0

0,5

1

1,5

2

2,5

3

3,5

4

MAE_ECWMF MAE_FUZZY BIAS_ECMWF BIAS_FUZZY

Error of temperature forecasts (ECMWF/FUZZY_system 1-10 days mean) winter 2003 30 stations

5.5.2003 44

Concluding remarks

• PPM system needs some tuning• After that it may be useful in editor environment

– As a new SmartTool-script– As a method within the editor

• Fuzzy system has to be studied further but the preliminary results look promising;

• However, Fuzzy system needs a lot of work compared to other methods

5.5.2003 45

Reference

Glahn, H.R.,1985: Statistical Weather Forecasting. Probability, Statistics and Decision Making in the Atmospheric Sciences, A.H. Murphy and R.W. Katz, Eds., Westview Press, 289-335.

R