Page 1© Crown copyright 2005 Use of EPS at the Met Office Ken Mylne and Tim Legg

© Crown copyright 2005 Page 1

Use of EPS at the Met Office

Ken Mylne and Tim Legg


Outline

Update on verification of First-Guess Early Warnings of severe weather

Example of unusual model and EPS behaviour

Met Office short-range ensemble development

Early Warnings of severe weather –The 4-day skill maximum investigated


Aims of this investigation

Verification of Early Warnings of Severe Weather presented last year showed a maximum in skill at day 4 for

EPS forecasts Deterministic forecasts from

T511 and EPS control Robust result but

ECMWF could not replicate Very little support in literature

Here we report further investigations of:

Can same result be replicated with Met Office model (UM)?

Definition of weather events.

D+1

D+2

D+4

D+5


Verification of Early Warnings

Early Warnings are verified against ‘Flash’

Warnings issued at short-range for the same

events (with a high degree of certainty)Warnings are verified on an “event” basis

An event can be a forecast event or an observed event

Each event is counted only once however long it lasts

This will be discussed further

Assessment period: 26 August 2003 to 29 April 2005.


Verification of UM-based Early Warnings ROC results (Heavy Rainfall)

Heavy Rainfall events

UM

UM results very similar to ECMWF T511 UM also has 4-day skill max

UM ROC Areas:•D+1 0.522•D+2 0.561•D+3 0.586•D+4 0.661•D+5 0.557•D+6 0.531


ROC results for Gales and Snow

Severe Gale events (Little evidence) Heavy Snowfall events (Similar to Rainfall)


Cost/Loss results (Heavy Rainfall)

Ensemble Control

T511 Met O U.M.


Spatial Averaging

Anders Persson suggested the 4-day skill max could be due to predictability on the spatial scale of the UKThis would suggest a shorter period max for smaller

regionsFGEW also gives probabilities for 12 sub-regions of

the UKVerification of these sub-region probs also show

the 4-day max We have not found any evidence to support this

idea but it does merit further investigation


The effect of how we define an ‘event’

There are two main ways of defining events for verification

(i) On an ‘event-wise’ basis – when an event occurs, did we have an early warning of it? And when an early warning exists, did an event occur? One contingency-table entry per event.

(ii) On a time-wise basis – at fixed time intervals, look to see whether or not an Early Warning and/or a Flash Warning were in force and complete contingency tables

Early Warnings have always been verified on an event basis:

This is different from most standard verification procedures which

use method (ii)

Could this account for the day 4 skill-max?


Definition of events for verification

Events are defined as:

An event can be a forecast event or an observed event

Each event is counted only once however long it lasts

Event spanning 2 days counted for 1st day only

Changing this to last day did not affect the day-4 skill max

For a warning to be correct (Hit), a warning and a verifying Flash

Warning must coincide for part of their validity period

Flash with no warning is a Miss

Warning with no Flash is a False Alarm

Correct Rejections defined for a complete 24-hour period with

no warning or Flash (except one already verified for previous

day)


Results for different event definitions

‘Event-wise’ ‘Time-wise’

for Heavy Rainfall warnings (01 Oct 2003 – 03 Nov 2004)


Conclusions on day-4 Skill Max

First-guess Early Warnings verification designed to assess the skill of warnings issued to end users:

Event-based Events of variable length Each event verified once only Precise timing not required for success – only some overlap

Latest results show that it is this definition which leads to the day-4 maximum in skill apparent in the results We do not claim to fully understand why! Could be related to

spatial averaging as suggested by Anders. We cannot assume that results from “standard” verification of

NWP will apply to user-oriented products based on NWP.email: [email protected] or [email protected]

Skill of EPS Control and EPS ModelExample raised by Met Office Chief

Forecaster


EPS Model

Forecaster was surprised and concerned that EPS control and T511 were so different at day 6 in this forecast, with only a difference in resolution

Opposite ends of EPS distributionIs this normal/ to be expected?Has the previously reported problem with the EPS model (time-stepping?) been solved?

CTRL T511

Short-Range Ensembles at the Met Office


Short-range Ensembles

ECMWF EPS has transformed the way we do Medium-Range Forecasting

Uncertainty also in short-range: Rapid Cyclogenesis often poorly forecast deterministically (eg Dec 1999) Uncertainty of sub-synoptic systems (eg frontal waves)

Many customers most interested in short-range Assess ability to estimate uncertainty in local weather

QPF

Cloud Ceiling, Fog

Winds etc THORPEX

Observation targeting Multi-model ensemble contribution

LBCs for future storm-scale ensembles


Ensemble Prediction Developments

Ensemble under development for short-rangeRegional ensemble over N. Atlantic and Europe (NAE)Nested within global ensemble for LBCsETKF perturbationsStochastic physicsT+72 global, T+36 regional

NAE


Progress with ETKF

ETKF set up with global UMProcessing all observations used in data

assimilation

12-hour cycle (f/c twice per day)

Running in conjunction with stochastic physics to propagate effect

Encouraging growth rate in case studies


Stochastic Physics Schemes

Three components to current stochastic physics: Installed in current version:

Stochastic Convective Vorticity (SCV)Random Parameters (RP)

Under test:Stochastic Kinetic Energy Backscatter (SKEB)

Page 29

SKEB

Stochastic Kinetic Energy Backscatter (SKEB) Based on original idea and previous work by Shutts (2004)

Closely related to ECMWF CASBS scheme

Aim: To backscatter (stochastically) into the forecast model some of the energy excessively dissipated by it at scales near the truncation limit

In the case of the UM, a total dissipation of 0.75 Wm-2 has been estimated from the Semi-lagrangian and Horizontal diffusion schemes. (Dissipation from Physics to be added later on)

Each member of the ensemble is perturbed by a different realization of this backscatter forcing

Page 30

SKEB

Streamfunction forcing:1

( , )2

DF K R

K.- Kinetic En.; R.- Random field;

D.- Dissipated en. in a time-step

R is designed to reproduce some statistical properties found with CRMs

Largest at the jets/storm track

Example: u increments at H500

Page 31

SKEB

Preliminary results: Positive increase in spread (comparable to that seen at ECMWF)

SKEB

RP+SCV

Increase in spread respect to an IC-only ensemble

500 hPa geopotential height


First Full Case Study Run (7-8 July 2004)

ETKF spun-up over 7 daysStochastic physics and ETKF interacting

Forecasts run to 5 daysSpread looks reasonable


T+48 Postage Stamps from January 8 2005 storm

AnalysisControl

Several members have better lowthan control. Member 4 is deeper. NB. This is global EPS.


Project Progress

Milestone: Implementation of demonstration ensemble based on NAE model for assessment by forecasters (August 2005)Global ensemble has now been running in our parallel

test suite for almost 2 weeks

NAE suite is nearly complete

Product generation and Verification systems are under development

We are on target to meet the milestone


Medium-Range (TIGGE)

Global forecasts will be extended to 15 days to contribute to THORPEX multi-model EPS research This will run at ECMWF using UK member state time

ETKF scheme believed suitable for Medium Range as well as Short Range Perturbations scaled to 12h forecast errors – could be

amplified if necessaryPlanned configuration

90km resolution 20 members twice per day


Conclusions

Closer to understanding 4-day skill max in severe weather warnings from EPSStandard NWP verification results may not always

translate to user-specific products and verification

Good progress with development of ensemble capability at the Met OfficeShort-range regional ensemble for EuropeContribution to global medium-range ensembles for

THORPEX

Documents

Page 1© Crown copyright 2005 Use of EPS at the Met Office Ken Mylne and Tim Legg