Applications of Ensemble Applications of Ensemble Prediction - a Historical Prediction - a Historical

PerspectivePerspective

Steve Tracton Office of Naval Research

Arlington, VA

(Formally of NWS/NCEP)

KEY POINTSKEY POINTS

•THERE ARE INEVITABLE UNCERTAINTIES IN NWP DUE TO UNCERTAINTIES IN INITIAL CONDITIONS AND MODEL FORMULATION

SCREAMING MESSAGE:

THERE WILL ALWAYS BE VARYING DEGREES OF UNCERTAINTIES IN FORECASTS (“Chaos Theory”)

KEY POINTSKEY POINTS• THERE ARE INEVITABLE UNCERTAINTIES IN NWP DUE TO UNCERTAINTIES IN

INITIAL CONDITIONS AND MODEL FORMULATION

• ENSEMBLE PREDICTIONENSEMBLE PREDICTION – FROM EARLY 90’S ON, – FROM EARLY 90’S ON, REVOLUTIONARY CHANGE IN THE THRUST OF OPERATIONAL REVOLUTIONARY CHANGE IN THE THRUST OF OPERATIONAL NWP (“WAVE OF THE FUTURE”), NWP (“WAVE OF THE FUTURE”),

First formal attention to the real possibility of OPERATIONALOPERATIONALensemble prediction at ECMWF Workshop on PredictabilityPredictabilityIn the Medium Range and Extended Range, 1986

Ultimately led to operational GLOBALGLOBAL EPS EPS at ECMWF and NMC in Dec 1992 Followed from Sufficient CPU resources becoming available Scientific basis for generating “dynamically constrained” initial state perturbations (SVs, Breeding) Development of output products

ENSEMBLE FORECASTING

Give Me Odds

KEY POINTSKEY POINTS• THERE ARE INEVITABLE UNCERTAINTIES IN NWP DUE TO UNCERTAINTIES IN

INITIAL CONDITIONS AND MODEL FORMULATION

• ENSEMBLE PREDICTIONENSEMBLE PREDICTION – FROM EARLY 90’S ON, – FROM EARLY 90’S ON, REVOLUTIONARY CHANGE IN THE THRUST OF OPERATIONAL REVOLUTIONARY CHANGE IN THE THRUST OF OPERATIONAL NWP (“WAVE OF THE FUTURE”), NWP (“WAVE OF THE FUTURE”), THE OBJECTIVES BEING THE OBJECTIVES BEING TOTO::

PROVIDE RELIABLE INFORMATION ON FORECAST PROVIDE RELIABLE INFORMATION ON FORECAST UNCERTAINTIES FROM THE SPREAD (DIVERSITY) AMONGST UNCERTAINTIES FROM THE SPREAD (DIVERSITY) AMONGST

ENSEMBLE MEMBERSENSEMBLE MEMBERS

NET RESULTNET RESULT - ENHANCE UTILITY/VALUE OF NWP FOR - ENHANCE UTILITY/VALUE OF NWP FOR VIRTUALLY ALL APPLICATIONSVIRTUALLY ALL APPLICATIONS

USE TO:USE TO:• Ascertain most likely deterministic prediction• Confidence in deterministic forecast• Same, plus identifying relative likelihood of alternative

scenarios

• Full probability distribution – maximum informationmaximum informationNOT NECESSARILY SKILL

NOGAPS

ECMWFMRF

UKMET

Providing EPS = Acceptance and Use /

RUN TO RUN MODEL CONTINUITY

GOOD

BAD

TODAY’S DAY 3 YESTERDAY’S DAY 4

NEW 84 HR MRF

OLD 108HR MRF

NEW 96 HRECMWF

OLD 120 HR ECMWF

WHICH SOLUTION IS PREFERRED? DETERMINISTIC THINKING

HPC’s Extended Forecast Discussion(released Mar 2, 2001, at 3:38 p.m.)

EPS NOW CONSIDERED INDISPENSIBLE TO HPC MEDIUM RANGE FORECASTS (Jim Hoke)

STEEP LEARNING CURVEFOR NEW PARADIGM:NO A-

PRIORI BEST SINGLE OUTCOME

“DETERMINISM IS DEAD” ??

NOT YET; BUT…

American Meteorological Society (AMS) Statement: Enhancing Weather Information with Probability Forecasts (3/02 BAMS!)

“The AMS endorses probability forecasts and recommends their use be substantially increased.”

“Would allow user (not forecaster!) to make decisions based on quantified uncertainties with resulting economic and

social benefits” ( e.g., from taking umbrella along, through canceling a trip ,to evacuation from an impending threat)

Schematic of how a probabilistic model forecast can be used for risk-based decision-making.

P1, P2 … PN = predicted variables of interest, for example, precipitation amount

Pcr = user-specified “critical value” of P which, if exceeded, requires an action or decision

T = chance of critical value being exceeded

Tcr = user-specified tolerance level (depending on societal, monetary, and/or environmental considerations).

P1

P2

P3

P4..PN

Model

F(P) Compare T with Tcr

Take appropriate action

Pcr

T

P

American Meteorological Society (AMS) Statement: Enhancing Weather Information with Probability Forecasts (3/02 BAMS!)

“The AMS endorses probability forecasts and recommends their use be substantially increased.”

“Would allow user (not forecaster!) to make decisions based on quantified uncertainties with resulting economic and

social benefits” ( e.g., from taking umbrella along, through canceling a trip, to evacuation from an impending threat)

“Probability Forecasts are particularly useful, even necessary,necessary, to reliably provide early warnings of extreme weather events”

TO AVOID, E.G.

Not Good- especially when effecting DC (just after announce-ment of new Super Computer by NWSHQ

MAJOR SNOWSTORM AMBUSHES WASHINGTON

FOR FORECASTERS, ANOTHER …FOR FORECASTERS, ANOTHER …

American Meteorological Society (AMS) Statement: Enhancing Weather Information with Probability Forecasts (3/02 BAMS!)

“The AMS endorses probability forecasts and recommends their use be substantially increased.”

“Would allow user (not forecaster!) to make decisions based on quantified uncertainties with resulting economic and

social benefits” ( e.g., from taking umbrella along through canceling a trip to evacuation from an impending threat)

Requires: Requires:

Conveying rationale and nature of uncertainties Conveying rationale and nature of uncertainties

Products/graphics/presentations that are readily Products/graphics/presentations that are readily comprehensible and relevantcomprehensible and relevant

KEY POINTSKEY POINTS• THERE ARE INEVITABLE UNCERTAINTIES IN NWP DUE TO UNCERTAINTIES IN INITIAL

CONDITIONS AND MODEL FORMULATION

• ENSEMBLE PREDICTION - REVOLUTIONARY CHANGE IN THE THRUST OF OPERATIONAL NWP (“WAVE OF THE FUTURE”) - CONSISTS OF MULTIPLE PREDICTIONS FROM SLIGHTLY DIFFERENT INITIAL CONDITIONS AND/OR WITH VARIOUS VERSIONS OF MODELS, THE OBJECTIVES BEING TO:

• REALIZING THE PRACTICAL UTILITY/VALUE OF ENSEMBLES REALIZING THE PRACTICAL UTILITY/VALUE OF ENSEMBLES ACCOMPLISHED VIA A VARIETY OF PRODUCTS DESIGNED ACCOMPLISHED VIA A VARIETY OF PRODUCTS DESIGNED TO CONDENSE AND MAXIMIZE INFORMATION CONTENT FOR TO CONDENSE AND MAXIMIZE INFORMATION CONTENT FOR USERSUSERS

Products/graphics/presentations must be readily comprehensible Products/graphics/presentations must be readily comprehensible and relevantand relevant

(MUST BE USER SPECIFIC AND USER FRIENDLY)

USER FEEDBACK USER FEEDBACK ESSENTIALESSENTIAL

Products: Stamp maps

H

H

H

H

H

L

L

Cntr T255 Cluster 1

H

H

H

H

H

L

L

L

L

L

OPER T511 Cluster 1

HH

H

HH

H

L

L

L

Verifying Analysis

MSLP (countour every 5hPa) and Temperature at 850hPa (only-6 and 16 isolines are plotted)Monday 26 May 2003 12UTC ECMWF Forecast t+168 VT: Monday 2 June 2003 12UTC Surface: mean sea level pressureECMWF ENSEMBLE FORECASTS

HH

H

H

L

Member 1 Cluster 1

H

H

HH

H

L

L

Member 2 Cluster 1

H

H

HH

L

L

Member 3 Cluster 1

H

H

H

H

L

L

L

L

Member 4 Cluster 1

H

H H

L

L

Member 5 Cluster 1

H

H

H

HH

L

L

L

L

Member 6 Cluster 1

H

H

H H

L 102

0

Member 7 Cluster 1

H

H

HH

Member 8 Cluster 1

H

H

H

H

LL

Member 9 Cluster 1

H

H

H

H

H

L

L

Member 10 Cluster 1

H

H

H H

L

L

101

5

Member 11 Cluster 1

H

HH

HH

L

Member 12 Cluster 1

H

H

LL

L

L

Member 13 Cluster 1

HH

H

H

L

L

Member 14 Cluster 1

HH

H

L

L

Member 15 Cluster 1

H

H

H

H

L

Member 16 Cluster 1

H

HH

H

HH

L

L

L

Member 17 Cluster 1

H

H

H

H

H

L

L

L

Member 18 Cluster 1

H

H H

L

L

Member 19 Cluster 1

H

H

H

H

L

LL

Member 20 Cluster 1

H

H

H

H

L

L

L

Member 21 Cluster 1

H

H

H

H

H

H

L

LL

Member 22 Cluster 1

HH

H

L

Member 23 Cluster 1

H

H

H

H

L

L

L

10

10

Member 24 Cluster 1

HH H

H

H

H

L

L

L

Member 25 Cluster 1

H H

H

H

H

L

Member 26 Cluster 1

H

H

HH

LL

L

Member 27 Cluster 1

H

H

HH

L L

Member 28 Cluster 1

H

H

H

H

H H

L

L

Member 29 Cluster 1

H

H

H

H

L

L

L

Member 30 Cluster 1

H

H

HH

H

Member 31 Cluster 1

H

H

H

H

LL

L

Member 32 Cluster 1

H

H

H

H

H

L

L

Member 33 Cluster 1

H

H

H

H

H

H

L

L

Member 34 Cluster 1

H

H

H

H

H

L

L L

L

Member 35 Cluster 1

H

H

H

H

L L

L

Member 36 Cluster 1

H

H

HH

L

L

Member 37 Cluster 1

H

H

H

LL

Member 38 Cluster 1

H

H

H

L

L

Member 39 Cluster 1

H

H

H

H

L

L

L

101

5

Member 40 Cluster 1

H

H

H

L

L

Member 41 Cluster 1

H

H

HH

L

L

Member 42 Cluster 1

H

H

H

H

H

L

L

Member 43 Cluster 1

H

H

H

H

L

L

L

L 101

5

Member 44 Cluster 1

H

H

H

H

HH

L

L

Member 45 Cluster 1

H

HH

L

L

Member 46 Cluster 1

H

H

H

H H

H

L

L

Member 47 Cluster 1

H

HH

LL

Member 48 Cluster 1

H

HH

L

L

L

L

Member 49 Cluster 1

H

H

HH

L

L

L

Member 50 Cluster 1

TYPES OF PRODUCTSTYPES OF PRODUCTS Roots of EPS products largely provided by Ed Epstein with graphical depictions to illustrate how uncertainty information could enhance forecast value (1971);

But, no further formalformal consideration of post processing and presenting EPS output until 1992 ECMWF Workshop on New Developments in Predictability (List of recommended generic products largely reflected “concept demonstration” mode of EPS experiments at NMC/CAC, Tracton and Kalnay, 1993)

SPAGHETTI CHARTSSPAGHETTI CHARTS: now one of the most recognizable and symbolic products of EPS

TYPES OF PRODUCTSTYPES OF PRODUCTS

SPAGHETTI CHARTS

MEAN/SPREAD PROBABILITIES STORM TRACKS CLUSTERS VERTICAL PROFILES METEOGRAMS ENSEMBLE DERIVED MOS

010519/0000V63 SREFX-CMB; LIFTED INDEX PROB 0F < -4

5-Day ECMWF Ensemble Prediction of Typhoon Rusa

TYPES OF PRODUCTSTYPES OF PRODUCTS

SPAGHETTI CHARTS

MEAN/SPREAD PROBABILITIES STORM TRACKS CLUSTERS VERTICAL PROFILES METEOGRAMS ENSEMBLE DERIVED MOS

CAN BE APPLIED TO VIRTUALLY ALL MODEL AND MODEL DERIVED CAN BE APPLIED TO VIRTUALLY ALL MODEL AND MODEL DERIVED PARAMETERS AND MODEL OUTPUT DRIVERS OF SECONDARY PARAMETERS AND MODEL OUTPUT DRIVERS OF SECONDARY

SYSTEMS, E.G., WAVE, HYDROLOGICAL,POLUTION DISPERSION SYSTEMS, E.G., WAVE, HYDROLOGICAL,POLUTION DISPERSION MODELSMODELS

ULTIMATELY CAN (MUST) BE PROPOGATED TO USER ULTIMATELY CAN (MUST) BE PROPOGATED TO USER SPECIFCIC SPECIFCIC QUANTITIES, E.G., UMBRELLA SALES, ENERGY USEAGE, TYPEQUANTITIES, E.G., UMBRELLA SALES, ENERGY USEAGE, TYPEOF “SMART” MUNITIONS, ETC., ETC., ETC., ETC., ETC., ETCOF “SMART” MUNITIONS, ETC., ETC., ETC., ETC., ETC., ETC ….

MANY NEW AND VARIED PRODUCTS POSSIBLE;

BUT, MOST SIGNIFICANT ADVANCE OVER PAST YEARS HAS BEEN EXPONENTIAL GROWTH IN GRAPHICS CAPABILITIES - ANIMATION, ZOOMING, 3-D, ETC. - AND PROLIFERATION OF PCs AND HIGH SPEED INTERNET

MANY NEW AND VARIED PRODUCTS POSSIBLE;

BUT, MOST SIGNIFICANT ADVANCE OVER PAST YEARS HAS BEEN EXPONENTIAL GROWTH IN GRAPHICS CAPABILITIES - ANIMATION, ZOOMING, 3-D, ETC. - AND PROLIFERATION OF PCs AND HIGH SPEED INTERNET

SO, REAL CHALLENGE IS MORE FULLY SO, REAL CHALLENGE IS MORE FULLY EXPLOITING POTENTIAL VALUE OF EPS IN EXPLOITING POTENTIAL VALUE OF EPS IN BOTH PUBLIC AND PRIVATE SECTORSBOTH PUBLIC AND PRIVATE SECTORS

CONVEYING UNCERTAINTY TO PUBLIC VIA MEDIA - A CONVEYING UNCERTAINTY TO PUBLIC VIA MEDIA - A CRITICAL LINK (see IABM.org) CRITICAL LINK (see IABM.org)

Bob Ryan

MON TUES WED THURS FRI

10

20

30

4036 28 23 21 31

40%

THIS

INCONSISTENT WITH PREDICTABILITY LIMITS

NOT THIS

Visualizing Uncertaintyin

Mesoscale Meteorology

APL Verification Methodology21 May 2002

Scott Sandgathe

Questions ??

SPAGHETTI CHARTSPAGHETTI CHART

What a mess!!

END

BACKUP SLIDES

EVEN EXPERTS CAN HAVE A BAD DAY

•One model (even with forecaster input) is an all or nothing proposition =>

“One detailed mesoscale model based

forecast could allow the user to make highly specific and detailed inaccurate forecasts.” (after Grumm)

•High Resolution Mesoscale models

–allow us to see features not in coarser models

–But: even small timing and placement errors can be significant in attempt to accurately forecast details (see Mass, et al., 3/02 BAMS!!!).

– But But: Forcaster judgement could mitigate

• Why we need ensembles

–Deal with uncertainties in analyses and model formulation

–But: Requires tradeoffs when computer resources limited (e.g., model resolution)

– But But: Mesoscale predictability often substantially controlled by synoptic predictability (and uncertainties therein)

=> Subjective or statistically based downscaling possible to get uncertainties in mesoscale weather

• Ideal: Ensembles with highest resolution justifiable

Compromise: Combination of single (or few) high resolution and coarser resolved ensemble

ADDITIONAL WEB SITES WITH ENSEMBLE PRODUCTS AND INFORMATION

Univ of Utah:http://www.met.utah.edu/jhorel/html/models/model_ens.html

FNMOC:http://152.80.49.204/PUBLIC/

Canada:http://www.cmc.ec.gc.ca/rpn/ensemble_products/index.html

CDC:http://www.cdc.noaa.gov/~map/maproom/ENS/ens.html

State College:http://bookend.met.psu.edu/ensembles/Ensemble.html

ENSEMBLE FORECASTING

Give Me Odds

EVEN EXPERTS CAN HAVE A BAD DAY

YOUR FORECAST HAS A 30% YOUR FORECAST HAS A 30% CHANCE OF BEING 70% CORRECTCHANCE OF BEING 70% CORRECT

Uncertain location of incoming western trough

From CDC web site: http://www.cdc.noaa.gov/map/images/ens/ens.html

Uncertain amplitudeof eastern trough

3-day forecast from 00 UTC 11/2/01, spaghetti diagram for ensembleglobal

SOME NCEP ENSEMBLE

MEMBERS OFFER A MORE “DIGGY”

CENTRAL U.S. TROF THAN THE OPERATIONAL

MRF..BUT NOT AS STRONG AS THE

ECMWF

“SPAGHETTI” DIAGRAM

DAY3

564 DM CONTOUR NCEP ENSEMBLE