Ensemble Prediction Systems and Probabilistic Forecasting Richard (Rick) Jones SWFDP Training...

Ensemble Prediction Systems and Probabilistic Forecasting

Richard (Rick) JonesSWFDP Training Workshop on Severe Weather Forecasting

Bujumbura, Burundi, Nov 11-16 , 2013Slides courtesy: UKMO Paul Davies and Chris

Tubbs

23-04-11 1

23-04-11 2

Computer models

Forecaster

Customer/Public

Atmosphere

Scientists

The forecaster misled me

The NWP misled me

Errorneous observations misled the NWP

The atmosphere is ”chaotic”

””The Blame Game” The Blame Game” or ”The Passing of The Buck”or ”The Passing of The Buck”

23-04-11 3

Understanding chaos

Numerical Weather Prediction

23-04-11 4

The Atmosphere

Model System

Data Assimilation

ForecastAnalysisError

Forecast distribution

DiagnosticTools and Products

23-04-11 5

Quantifying uncertainty with ensembles

time

Forecast uncertainty

Climatology

Initial Condition Uncertainty

X

Deterministic Forecast

Analysis

CHAOS

April 2013 Anders Persson 6

-12 0 +12 +24 +36 +48 +60 +72 +84 h

ψ

Exchanging the “accurate” forecast with a more “honest” one

Dangerous threshold

Today’s NWP forecast

Today’s EPS forecast

Anders Persson 711/04/23 7

ψ

Correction for systematic errors

-12 0 +12 +24 +36 +48 +60 +72 +84 h

Anders Persson 811/04/23 8

ψ

●●

●

● ●

●●●

●

●

●●

obs ●● ●

The final ensemble forecast – with verification

70% 50%

-12 0 +12 +24 +36 +48 +60 +72 +84 h

Anders Persson 9

Ψ

Probability

Investigations show that “jumpiness” correlates badly to the accuracy of the last forecast

NWP today

NWP yesterdayNWP the day

before yesterday

We might form an opinion by looking at the last NWPs from the same model, so called “lagged” forecasts

Anders Persson 1011/04/23

Medium range forecasting…with deterministic and EPS information

Latest three NWP

Latest EPS

Most common case with good agreement between EPS

spread and NWP “jumpiness”

Most common case

Anders Persson 1111/04/23

Rather poor agreement between larger EPS spread and small NWP

“jumpiness”.

The analysis system has obviously managed to avoid possible

problems because the NWP is not very “jumpy”

Should the forecasters be more certain than the EPS indicates?

Rather common case

Anders Persson 1211/04/23

Rather poor agreement between small EPS spread

and large NWP “jumpiness”.

The perturbations have not been quite able to cover the

analysis uncertainties

Should the forecasters be more uncertain than the EPS indicates?

Not uncommon case

Anders Persson 1311/04/23

Poor agreement between the main directions of the EPS

and the NWP

This puts the forecasters in a very difficult situation and there is not enough experience or investigations about this situation

Rare case

Best choice: create a “super ensemble”

See TIGGE

The Effect of Chaos

23-04-11 14

Tiny errors in our analysis of the current state of the atmosphere lead to large errors in the forecast – these are both equally valid 4-day forecasts.

• We can usually forecast the general pattern of the weather up to about 3 days ahead.

• Chaos then becomes a major factor

• Fine details (eg rainfall) have shorter predictability

Ensembles• In an ensemble forecast we run the model many times from slightly different initial conditions• This provides a range of likely forecast solutions which allows forecasters to:

– assess possible outcomes;

– estimate risks – gauge confidence.

23-04-11 15

23-04-11 16

Reminder on scales and predictability

Temporal Resolution

23-04-11 17

Hail shaft

Thunderstorm

Front

Tropical Cyclone

Space Scale

Lifetime

Predictability?

10mins 1 hr 12hrs 3 days

30mins 3 hrs 36hrs 9 days

1000km

100km

10km

1km

MCS

Short-range Ensembles

ECMWF EPS has transformed the way we do Medium-Range Forecasting

• Uncertainty also in short-range:– Rapid Cyclogenesis often poorly forecast deterministically– Uncertainty of sub-synoptic systems (eg thunderstorms)– Many customers most interested in short-range

• Assess ability to estimate uncertainty in local weather– QPF– Cloud Ceiling, Fog – Winds etc

23-04-11 18

Initial conditions perturbations

• Perturbations centred around 4D-Var analysis• Transforms calculated using same set of

observations as used in 4D-Var (including all satellite obs) within +/- 3 hours of data time

• Ensemble uses 12 hour cycle (data assimilation uses 6 hour cycle)

23-04-11 19

Initial conditions perturbationsDifferences with ECWMF Singular Vectors:

- It focuses on errors growing during the assimilation period, not growing period:

- Suitable for Short-range!

- Calculated using the same resolution than the forecast

- ETKF includes moist processes

- Running in conjunction with stochastic physics to propagate effect

23-04-11 20

23-04-11 21

Model error: parameterisations

Random parameters

Parameter Scheme min/std/MaxEntrainment rate CONVECTION 2 / 3 / 5

Cape timescale CONVECTION 30 / 30 / 120

RH critical LRG. S. CLOUD 0.6 / 0.8 / 0.9

Cloud to rain (land) LRG. S. CLOUD 1E-4/8E-4/1E-3

Cloud to rain (sea) LRG. S. CLOUD 5E-5/2E-4/5E-4

Ice fall LRG. S. CLOUD 17 / 25.2 / 33

Flux profile param. BOUNDARY L. 5 / 10 / 20

Neutral mixing length BOUNDARY L. 0.05 / 0.15 / 0.5

Gravity wave const. GRAVITY W.D. 1E-4/7E-4/7.5E-4

Froude number GRAVITY W.D. 2 / 2 / 4

QUMP (Murphy et al., 2004)Initial stoch. Phys. Scheme for the UM (Arribas, 2004)

Using probabilities

• Recipients of forecasts & warnings are sensitive to different levels of risk: reflecting cost of mitigation vs expected loss

• An intelligent response to forecasts & warnings depends on risk analysis, requiring knowledge of impact probability

• Use of ensembles to estimate probability at longer lead times is well established in meteorology

23-04-11 22

WMO SWFDP Macau 10 April 2013 Anders Persson

23

10 20 30 40 50 Ψ

Median 50-50%

25%=12-13 forecastsΨ= 31-37

25%=12-13 forecastsΨ= 23-30

25%=12-13 forecastsΨ= 10-22

25%=12-13 forecastsΨ= 38-50

10 20 30 40 50 Ψ

0% 25% 50% 75% 100%

| | ||| ||| || | | | ||| | | || || ||| |||| || || || | | | | | ||| | || | | | |

23-04-11 24

Probability maps

23-04-11 25

23-04-11 26

EC Total ppn prob > 20mm 12z Tue – 12z Wed

23-04-11 27

EPSgrammes

23-04-11 28

Total cloud cover

6 hourly precipitation

10m wind speed

2m temperature

EPS control

Deterministic

median (50%)

max

min

90%

10%

75%

25%

Bujumbura 06 Nov 12UTC

23-04-11 30

23-04-11 31

23-04-11 32

The Extreme Forecast Index (EFI)

Extreme forecast index (EFI)

• EFI measures the distance between the EPS cumulative distribution and the model climate distribution

• Takes values from –1 (all members break climate minimum records) and +1 (all beyond model climate records)

• The main idea is to have an index that can be conveniently mapped – removing the effect from different climatologies – to use as an “alarm bell”

23-04-11 33

EFI

• Experience suggests that EFI magnitudes of 0.5 - 0.8 (irrespective of sign) can be generally regarded as signifying that "unusual" weather is likely whilst magnitudes above 0.8 usually signify that "very unusual" or extreme weather is likely. Although larger EFI values indicate that an extreme event is more likely, the values do not represent probabilities as such

23-04-11 34

23-04-11 35

0%

EPS distributionClimate

distribution

Clim.mean

Temperature

EPSmean

Advantages with probability density functionsMeans and asymmetric variances are easily spotted

100%

23-04-11 36

0%

100%

EPS distributionClimate

distribution

Clim.mean

Temperature EPSmean

Advantages with probability density functionsMeans and asymmetric variances are easily spotted

23-04-11 37

EFI ~ 70%

EFI ~ 70%

The EFI generally does not take the probability into account

23-04-11 38

EFI ~ -50%

EFI ~ 50%

For temperature the EFI can take values < 0

EFI 2m Temp

23-04-11 39

EFI 10m gusts

23-04-11 40

EFI wind speed

23-04-11 41

EFI precip

23-04-11 42

Other products

23-04-11 43

Tropical strike probability

23-04-11 44

Tropical strike

23-04-11 45

Ensemble mean

23-04-11 46

Forecast Probability

23-04-11 47

UKMO African LAM

23-04-11 48

UKMO African LAM

23-04-11 49

Ensemble forecasts

• Ensemble forecasts help us

1. To judge the (un)certainty of the weather situation

2. To acquire probability estimates of anomalous events (extreme or high impact)

3. To get the most accurate and least “jumpy” deterministic forecast value

23-04-11 50

23-04-11 52

Questions & Answers