Extreme weather and probabilistic forecast approaches

Mesoscale Weather PredictionEnsemble Post-Processing

Prediction and VerificationResults

Extreme weather and probabilistic forecast

approaches

Petra Friederichs

Sabrina Bentzien, Andreas Hense

Meteorological Institute, University of Bonn

Statistical Methods for Meteorology and Climate Change

Montreal, 12 – 14 January 2011

P.Friederichs Extreme weather 1 / 33



Outline

Extreme weather and probabilistic prediction

I Atmospheric scales and mesoscale atmospheric dynamics

I High-impact weather

I Ensemble prediction systems

Ensemble post-processing

I Post-processing: Extract and calibrate information

I Verification

Results




What are extemes?

I Mathematically:

Defined as block maxima or excedances of large thresholds.

Events that lie in the tails of a distribution

I Perseption:

Rare, exceptional, ”large” and high impact

I Problems:

I 95% quantile of daily precipitation: ≈ 10− 15mm/d

I ≈ 2 years of data – only few extremes events for verification




Mesoscale ExtremesPredictabilityCOSMO-DE Ensemble Prediction System

Mesoscale Weather Prediction

I Strong and disastrous impact of many weather extremes calls

for reliable forecasts

I ”Although forecasters have traditionally viewed weather

prediction as deterministic, a cultural change towards

probabilistic forecasting is in progress.” (T N Palmer, 2002)

I Weather extremes do not come ”Out of the Blue”

I Numerical weather forecast models provide reliable forecasts

of the atmospheric circulation prone to generate extremes

I Combination of dynamical and statistical analysis methods





Atmospheric scales and mesoscale dynamics

I Different scales exhibit different

dominant force balances,

different wave dynamics

I Mesoscale on horizontal scales

2km – 2000km

I Complex force balances

Steinhorst, Promet 35, 2010





Mesoscale weather extremes

I Heavy thunderstorms on July 14, 2010

I Strong horizontal gradients

I Strong vertical mixing

I Embedded in larger scale squall line –

embedded in synoptic situation





Connection of Extremes on Different Scales

I Large vertical gradients of

entropy

I Convective instability

I Deep convection lead to

extremal vertical velocities

I Heavy precipitation and

hailstones grow within this

vertical circulation

Koeln 04.07.1994 RR=11,723mm

Zeit in min

Hoe

he in

km

0.2

0.2

0.4

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

2

0 10 20 30 40 50 60 70 80 90

0

1

2

3

4

5

6

7

8

9

10

0

10

20

30

40mm/h

g/kg

0

1

2

3

4

S. Bentzien (2009)





Predictability

I Inherent limit of predictability

I Fastes error growth at smallest

scales

I Predictability strongly depends

on flow regime

I Moist convection is primary

source of forecast-error growth

I Mesoscale forecasts are issued

for ≤18h-24h

E N Lorenz, Tellus 21, 19 (1969)





COSMO-DE Ensemble Prediction System (EPS)

I COSMO-DE: 2.8 km grid spacing,

convection resolving NWP model

I Operational forecasts 0-21 hours –

high-impact weather by DWD

I EPS with 20 (40) members

I Uncertainty due to initial conditions,

boundary conditions, and model

parameterisation errors

I First EPS with convection resolving

limited area NWP model

COSMO-EU

COSMO-DE

GME

Initial State Boundary Model

S. Theis, DWD (2010)




Censored Quantile RegressionMixture GLMExtrem Value TheoryMixture GLM including Extremes

Probabilistic forecasting: Maximize sharpness of

the predictive distribution subject to calibration

from Hamill (2006)

Calibration:

Raw ensemble data need adjustmend: biased and underdispersive

Gneiting et al. (2005)

Sharpness: Information of forecasts





Conditional quantile function

Semi-parametric

I A-priori probability τ , estimate

conditional quantile F−1Y |X(τ |x) = βT

τ x

via (linear) quantile regression

Parametric

I A-priori assumption about parametric

distribtion FY |X(y |x) = G (y ; Θ(x))

Estimate parameter function Θ(x)

Generalized linear model (GLM)





Quantile regression

QZQR(τ |X) = βT

τ X, βTτ = (β0, . . . , βK )

βτ = arg minβτ

n∑i=1

ρτ

(yi − βT

τ xi

)Censored quantile regression

QZQR(τ |X) = max(0,βT

τ X), βTτ = (β0, . . . , βK )

βτ = arg minβτ

n∑i=1

ρτ

(yi −max(0,βT

τ xi ))

with ρτ (u) = τu for u ≥ 0 and ρτ (u) = (τ − 1)u for u < 0





Censoring

Equivariance with respect to non-decreasing function h(·)Qh(Y )(τ) = h(QY (τ))

Hidden process Y ∗ observed through censored variable Y

Y = h(Y ∗) = max[0,Y ∗]

QY ∗QR

(τ |X) = βTτ X

0.1 0.2 0.3 0.4 0.5

−20

020

4060

meanmedian

QYQR(τ |X) = max(0,βT

τ X)

0.1 0.2 0.3 0.4 0.5

−20

020

4060

80 median





Generalized linear model – Mixture model

I Probability of precipitation (Y ≥ 0.1mm) – Logistic regression

Pr(Y ≥ 0.1 | x) = π(x)

I Distribution of precipitation – Gamma GLM

F (Y | x,Y ≥ 0.1) = GΓ(Y ; Θ(x))

Conditional mixture model for precipitation

FYmix(y | x) = (1− π) + π GΓ(y ; Θ(x)) Iy≥0.1





Conditional mixture model

Parametric – ’normal’

I A-priori assumption about parametric

distribtion for ’normal’ part

FY |X(y |x) = G (y ; Θ(x))

Estimate parameter function Θ(x)

Generalized linear model (GLM)

Parametric: ’extreme’

I Above threshold/quantile: parametric

distribtion FY |X(y |x) = G (y ; Θ(x)) is

of the familily of max-stable

distributions.





Extreme value theory ”Going beyond the range of the data”

I Limit theorem for sample maxima

→ asymptotic distribution for extremes

I Condition of max-stability (de Haan, 1984)

→ maxima follow a generalized extreme value

distribution

I Garantees universal behavior of extremes

→ enables extrapolation!

In praxis: often not enough data to reach asymptotic limit





Extreme value distribution

Generalized extreme value distribution (GEV)

Gξ(y) =

exp(−(1 + ξ y−µσ )−1/ξ)+, ξ 6= 0

exp(− exp(− y−µσ )), ξ = 0

,

Gumbel (..., shape=0.0)

PD

F

−2 0 2 4 6

0.0

0.1

0.2

0.3

0.4

Frechet (..., shape=0.6)

PD

F

−2 0 2 4 6

0.0

0.1

0.2

0.3

0.4

Weibull (..., shape=−0.3)

PD

F−2 0 2 4 6

0.0

0.1

0.2

0.3

0.4





Generalized Pareto distribution

Threshold excesses Z = Y − u follow a GPD

Prob(Z ≤ y − u) =

1−

(1 + ξ y−u

σu

)−1/ξ, ξ 6= 0

1− exp(− y−u

σu

), ξ = 0

,

for y − u > 0 and (1 + ξ y−uσu

) > 0





Mixture GLM including Extremes

I Conditional mixture model for precipitation

FYmix(y | x) = (1− π) + π GΓ(y ; Θ(x)) Iy≥0.1

I GPD for ’extreme’ precipitation – above uτ = F−1Y (τ | x)

F (Y | x,Y > uτ ) = GGPD(Y − uτ ;σ(x))

Conditional mixture model for precipitation with extremes

FYGPD(y | x) = (1− π) + π GΓ(y ; Θ(x)) IY≥0.1,Y≤uτ

+ (1− τ)GGPD(Y − uτ ;σ(x) IY>uτ




Proper Scoring RulesForecasts in terms of quanitles

Prediction and Verification

I Model parameter training

I Verification on independent data

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Jul.08 Okt.08 Jan.09 Apr.09 Jul.09 Okt.09 Jan.10 Apr.10

n.train20,30,..,90 Tage

30T.





Forecast verification by means of scores

I Cost functions or distance between forecast and data

I Utility measure in a Bayesian context

A score is proper iff

Ey∼Q [S(P, y)] ≥ Ey∼Q [S(Q, y)] ∀ P 6= Q

S(P, y): score function

Q forecasters best guess

Ey∼Q [S(., y)] expectation of S(., y) over y ∼ Q





Verification: Goodness-of-fit criterion

QVS(τ) = min{β∈Rq}

∑i

ρτ (yi−βT xi ) QVSref (τ) = min{β0∈R}

∑i

ρτ (yi−β0)

Quantile verification skill score QVSS(τ) = 1− QVS(τ)

QVSref (τ)

Log-likelihood ratio test: asymmetric Laplacian regression

fτ (u) =τ(1− τ)

σLexp(−ρτ (u)/σL).

proportional to log(QVS(τ)/QVSref (τ))




Ensemble Post-ProcessingVerificationExampleConclusions and Challenges

SKeleton EPS Interim Solution – Neighborhood method

I ’Time-lagged’ ensemble of

COSMO-DE

I Initialized every 3 h,

forecasts for 21 h

I 1 July 2008 – 30 April 2010

0 3 6 9 12 15 18 21 Vorhersagestunde

I First guess probability (fgp)

I First guess 0.9-quantile

(fgq9)

I 4 COSMO-DE forecasts and

5× 5 neighborhoodlagged average ensemble

forecast (LAF)

• Umgebung

• 4x9 member

• 4x25 member

• ...

6





12h accumulated precipitation between 12 and 00 UTC

I 83 Station in NRW

I Rain radar composite

6 7 8 9

50.5

51.0

51.5

52.0

52.5

longitude

latitude

0

200

400

600

800

1000

1200

1400





Reliability

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

Forecast probability, yi

Obs

erve

d re

lativ

e fr

eque

ncy,

o1

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

LR:fgpLR:fgp+q9

020000040000060000080000010000001200000

020000040000060000080000010000001200000

●

●

●

●

●

●

● ●

●

●

0 20 40 60 80

0

20

40

60

80

0.9 quantile

quantile forecast

empi

rical

qua

ntile

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

Ensemble QRmix.mod transf.fgq9

0 20 40 60 80

0.0

0.2

0.4

0.6

0.8

0.9 quantile

bins

freq

uenc

y





Quantile forecasts - Skill Score

6 7 8 9

50.5

51.0

51.5

52.0

52.5

longitude

latit

ude

0.0

0.1

0.2

0.3

0.4

0.5

0.6

6 7 8 9

50.5

51.0

51.5

52.0

52.5

longitude

latit

ude

0.0

0.2

0.4

0.6

0.8






●

●

●

●●●●

●●●●●●

●●●●●●●●●

●●●●●●●●●●●●

●●●●●●

●●●●●●

●●●●●●

●●●●●●●●●●

●●●●●●●●●

●●●●●●●

●●●

●●

0.99 quantile

QV

SS

0 10 20 30 40 50 60 70 80QR MIX MIX.t GPD

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

● QRMIXMIX.tGPD+

+

+++++

+

+

++

+

+

+

++++

+

+

++

+

+++

++++++++

+

+

+

++

+

++

+++

+

+

+++

+++

+

+++

++++

+

+

++

+

+

+

+

+

+++

++++

+

+++

+

+

+

+

+++

++

+

+

+

+

++

+

++

++

+

+

+++++++

+

++

++++

+

+

+

++

+++

+

+

+

+

+

++

+

+++

+

++

+

+++++

+

++

+

+

+

+

+

+++

+

+++

+

++

+

+

+

●

++

+++++

+

+

++

+

+

+

++++

+

+

++

+

+++

++++++++

+

+

+

++

+

++

+++

+

+

+++

+++

+

+++

++++

+

+

++

+

+

+

+

+

+++

++++

+

+++

++






●●●

●●●

●●●●●●●●

●●●●●

●●●●●●●●●●

●●●●●●●

●●●●●●●●

●●●●●●●●●●●●●●●●

●●●●●●●●●

●●●●●●●●●●●

●

●●

0.99 quantile

QV

SS

0 10 20 30 40 50 60 70 80QR MIX MIX.t GPD

−0.3

−0.2

−0.1

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

● QRMIXMIX.tGPD

++++

+

++

+++

++

+

+

+

++

++

+

+

+

+

+

+

+

+

+

+

+++

++++

+

+

+

+

+

++

++

+

+

++

++

++

++

++

+

+

++

+

+

++

+

+

+

+++++

+

++++++

++

+

++++

+

+

+

+++

++

+

+

+

++

+

+

++

+

+

++

+

+

+

+

+++

++++

+

+

+

++

++

++

+

+

++

++

++

++

++

+

+

++

+

+

++

+

+

+

+++++

+

++++++

++

+

++++

+

++

+++

+

+

+

++

++

+

+

++

+

+

++

+

+

+

+

+++

++++

+

+

+

+

+

++

+++

+

++

++

++

++

++

+

+

++

+

+

++

+

+

+

+++++

+

++++++

++

+





● ● ●

●

● ●●

● ●●

● ●

●

● ● ● ● ● ●

●

●

●

●

● ● ● ● ● ● ● ● ● ●

●

● ● ● ● ● ●0

10

20

30

40

50

Kleve

mm

/12h

Jul 05 Jul 15 Jul 25 Aug 04

q0.75

q0.90

q0.95

q0.99● station

radarQRMIXMIX.tGPD

● ● ●

●

●●

●

● ●●

●●

●●

●● ● ● ●

●

●

●

● ●●

●● ● ●

●

● ● ●●

● ● ● ● ● ●0

10

20

30

40

50

60

70Breckerfeld−Weng (Ennepe−Ruhr−Kreis)

mm

/12h

Jul 05 Jul 15 Jul 25 Aug 04

q0.75

q0.90

q0.95

q0.99● station

radarQRMIXMIX.tGPD





Lagged ensemble forecasts and radar measurements

6 7 8 9

50.5

51.0

51.5

52.0

52.5

longitude

latit

ude

20

40

60

80

6 7 8 9

50.5

51.0

51.5

52.0

52.5

longitude

latit

ude

20

40

60

80

6 7 8 9

50.5

51.0

51.5

52.0

52.5

longitude

latit

ude

50

100

150

6 7 8 9

50.5

51.0

51.5

52.0

52.5

longitude

latit

ude

20

40

60

80

6 7 8 9

50.5

51.0

51.5

52.0

52.5

longitude

latit

ude

20

40

60

80

6 7 8 9

50.5

51.0

51.5

52.0

52.5

longitude

latit

ude

20

40

60

80





Conclusions

I Weather forcasts provide information that conditions occurence of

extremes

I Linear (non-linear) statistical modeling extracts information

I Extreme value theory provides distributions tailored for extremes

I Parametric method less uncertain than non-parametric method and

non-linear dependecy (shape parameter) is not parsimony

I High-impact weather: insufficient data available for training and for

validation





Challenges

I Improve physical understanding of generation processes of extremes

I Application to multi-variable and spatio-temporal predictions

I Combine spatial statistics with model post-processing

(Berrocal et al., 2007)

I Develop methods for multivariate post-processing

I Develop novel ensemble methods tailored to extremes

(Bayesian model averaging)

I Verification tailored to extremes

I Verification for probabilistic multivariate and spatial forecasts





Reference

I Gneiting, T., A. E. Raftery, A. H. Westveld, and T. Goldman, 2005: Calibrated

probabilistic forecasting using ensemble model output statistics and minimum

CRPS estimation. Monthly Weather Review, 133, 1098-1118.

I C. Gebhardt, S.E. Theis, M. Paulat, and Z. Ben Bouallegue, 2010: Uncertainties

in COSMO-DE precipitation forecasts introduced by model perturbations and

variation of lateral boundaries. Atmos. Res. (in press).

I Friederichs, P., 2010: Statistical downscaling of extreme precipitation using

extreme value theory. Extremes 13, 109-132.

Special thanks to:

Susanne Theis, Martin Gober, Deutscher Wetterdienst, Offenbach

Thank You for Your Attention!


Documents

Extreme weather and probabilistic forecast approaches