2.3.3 ADVANCES IN INTENSITY GUIDANCE Rapporteurs: Buck Sampson (NRL – USA) and John Knaff (NESDIS – USA) Working group: Joe Courtney (BOM - AUS), Brian

2.3.3 ADVANCES IN INTENSITY GUIDANCE

Rapporteurs: Buck Sampson (NRL – USA) and John Knaff (NESDIS – USA)

Working group: Joe Courtney (BOM - AUS), Brian Strahl (JTWC – USA), Tsukasa Fujita (JMA – Japan), Naohisi Koide (JMA – Japan), Olivier Bousquet (METEO-FRANCE/LACY),Thierry Dupont (METEO-FRANCE/DIRRE/CYC – La Reunion), Mike Brennan (NHC – USA), Vijay Tallapragada (NCEP – USA), Tim Marchok (GFDL – USA), Sundararaman G. Gopalakrishnan (HRD – USA), Baode Chen (CMA – PRC), M. Mohapatra (IMD-India), S. D. Kotal (IMD-India) and U. C. Mohanty (IMD – India), Mike Fiorino (ESRL – USA), Jim Doyle (NRL – USA), Russ Elsberry (NPS – USA)

Jeju Island, Korea December 2014IWTC VIII

Eighth WMO International Workshop on Tropical Cyclones2

Guidance history in terms of MAE

Track MAE reduced dramatically over last 15 years

Official forecasts have improved very little

Intensity guidance has improved 1% to 2% per year (MAE)

Very recently intensity guidance is beginning to improve the official forecasts

N. West Pacific

S. Hemisphere

DeMaria et al. (2014)


How intensity forecasts are made…


NHC Forecasting

Perform analysis Dvorak, ADT, A/C, dropsondes,

SFMR, radar, AMSU, scatt

Display aids (see graphic) Use global models for

environmental factors Intensity trends, extratropical

transition

Check ocean fields SST, Ocean Heat Content, land

interaction

Use other guidance SHIPS/LGEM, consensus, SHIPS-RI


La Reunion Forecasting

Perform analysis Dvorak, …

Base forecast mostly on human expertise

Check global environmental factors Vertical windshear, low/mid- level

humidity, upper level divergence, outflow patterns, low level inflow

Intensity trends, extratropical transition

Check ocean fields SST, ocean heat content fields,

land interaction Maximum potential intensity

Use objective aids and NWP V and MSLP trends,

ensembles, STIPS

Current domain of Arome-OI (yellow) overlying ALADIN-Réunion operational domain (area of responsibility of RSMC La Réunion; white)


Tokyo Forecasting

Do Analysis Dvorak, …

Check global environmental factors e.g. vertical windshear

Check ocean fields SST, ocean heat content fields

Use objective guidance Climatic guidance, global

atmospheric model


New Delhi Forecasting

Do analysis Satellite, radar, …

Check NWP models 11 of them!

Check statistical models SCIP (like SHIPS) RI probabilities Landfall model for slow

decay over flat land

Use SCORPIO web page


BoM Forecasting

Do analysis Satellite, radar, …

Do NWP model assessment How are they performing? Are they consistent?

Check environmental factors Vertical wind shear, upper

level flow, low- to mid-level moisture, low-level inflow, proximity to land

View guidance STIPS, SHIPS/LGEM Consensus

Check forecast consistency


JTWC Forecasting

Perform analysis Satellite, radar, environment, … One Dvorak T-no/day

Do NWP model assessment How are they performing? Are they consistent?

Check environmental factors Vertical wind shear, upper level

flow, outflow channels, low- to mid-level moisture, low-level inflow, proximity to land, SST, OHC, TUTT

Check internal factors Eyewall replacement, annular

Check statistical aids, consensus STIPS, SHIPS/LGEM, GPCE

Look for forecast consistency


Forecasting Similarities

Analysis is important Environmental factors explored using global models

Vertical wind shear, upper level flow, outflow channels, low- to mid-level moisture, low-level inflow, proximity to land, SST, OHC

Subjective analysis still plays large role Statistical models are still important Mesoscale and regional models are becoming more

skillful RI guidance is still limited


Guidance

Statistical models Statistical-dynamical models NWP models Ensembles Consensus Probabilistic guidance Issues


Statistical Models

Served as short range guidance in early years May be used as guidance in longer ranges Serve as skill baselines for other methods Examples: Intensity CLIPER (e.g., ST5D, SHF5)

persistence, analog (e.g., WANI)


Statistical-Dynamical Models

Regression based on parameters we know matter Been around since 1991 Show skill relative to Intensity CLIPER out to about 72 h Upgraded to include many parameters with incremental

increases in skill Basis for other guidance (e.g., RI, annular storms,

eyewall replacement cycles) Examples:

SHIPS LGEM STIPS SCIP


Dynamical Models

Now routinely available and skillful! Most effort spent, most improvement obtained http://www.emc.ncep.noaa.gov/HWRF/IWTC_VIII/IWTC.html


Global Examples

Global ModelsModel Name Information NotesARPEGE (France;Action de Recherche Petite Echelle Grande Echchelle)

http://www.cnrm-game.fr/spip.php?article121

GFS (USA;Global Forecast System)

http://www.emc.ncep.noaa.gov/GFS/doc.php

GDPS(Canada; Global Deteministic Prediction System)

http://weather.gc.ca/model_forecast/global_e.html

Met Office(UK; Met Office)

http://research.metoffice.gov.uk/research/nwp/numerical/operational/

Also known as the unified model

NAVGEM (USA; NAVy Global Environmental Model)

http://www.nrlmry.navy.mil/metoc/nogaps/ Replaces NOGAPS

NGFS (India:GFS run at NCMRWF)

http://www.ncmrwf.gov.in/#

ECMWF (Europe; European Center for Medium Range Weather Forecasts)

http://www.ecmwf.int/en/forecasts/documentation-and-support

ACCESS (Australia; Australian Community Climate and Earth-System Simulator)

http://www.bom.gov.au/australia/charts/about/about_access.shtm

GSM (Japan; Global Spectral Model) CMA-GSM(China; Global Spectral Model run at NMC/CMA)

http://www.jma.go.jp/jma/en/Activities/nwp.html Yu et al. (2013)

http://www.jma.go.jp/jma/en/Activities/nwp.html


Regional ExamplesRegional Models

Model Name Information NotesGFDL (USA; Geophysical Fluid Dynamics Lab)

http://www.gfdl.noaa.gov/operational-hurricane-forecasting

Running in West Pacific

GFDN (USA; run off of NAVGEM by the US Navy)

Development behind GFDL

ALADIN (France/Consortium)

http://www.cnrm.meteo.fr/aladin/?lang=en

AROME-IO(France; AROME-Indian Ocean)

http://www.cnrm.meteo.fr/spip.php?article120

AROME will replace ALADIN in 2015

HWRF (USA; Hurricane Weather and Research Forecast Model)

http://www.emc.ncep.noaa.gov/?branch=HWRF

Coupled,3km inner meshAlso run in W. Pacific and SH

NHWRF (India; Hurricane Weather and Research Forecast Model run at NCMRWF)

COAMPS-TC (USA)

http://www.nrlmry.navy.mil/coamps-web/web/tc?&spg=2&scl=3

Coupled, big improvements in 2014 thus far.

ACCESS-TC(Australia; TC centric)

http://www.bom.gov.au/australia/charts/about/about_access.shtml

Replaces TC-LAPS, 3 re-locatable domains

ACCESS-R (Australia; Regional)

http://www.bom.gov.au/australia/charts/about/about_access.shtml

GRAPES-TCM (China; GLOBAL/REGIONAL ASSIMILATION AND PREDICTION SYSTEM –TC Model) GRAPES-TCM (China; GLOBAL/REGIONAL ASSIMILATION AND PREDICTION SYSTEM –Tropical Model)

http://www.ral.ucar.edu/hurricanes/repository/models/grapestcm.php Yu et al. (2013)

http://www.ral.ucar.edu/hurricanes/repository/models/grapestcm.php




GFDL Improvement


H-WRF Improvement

Left: Non-homogenous comparison of the absolute track forecast errors between the 2012 HWRF version during 2012 (blue columns) and the 2013 HWRF version during 2013 (red columns), Right: similar to (a) but for the absolute intensity forecast errors.


AROME Improvement

Intense TC Béjisa intensity (central pressure, HPa) simulated by ECMWF (red), AROME OI (brown) and AROME-OI coupled with an oceanic model (blue) NWP systems. Left panel: intensity analysis (left panel) and 72-h intensity forecast starting at 12 UTC, 30 Dec 2013 (centre panel) and 12 UTC, 02 January 2014 (right panel). The best track produced by RSMC La Réunion is shown in black.


IMD H-WRF

Mean intensity forecast error (kts) of HWRF during 2013 (total number of forecasts verified is shown in the bracket)

21.9

16.1

9.811.1

13.3

18.3

25.4

18.3

26.3

43

23.1

17.4

11.4 11.915.1

20.8

26.4

18.5

26.5

43

0.0

5.0

10.0

15.0

20.0

25.0

30.0

35.0

40.0

45.0

50.0

12(23) 24(21) 36(18) 48(16) 60(12) 72(10) 84(8) 96(6) 108(4) 120(2)

Forecast Lead Time(h)

Inte

nsi

ty fo

reca

st e

rro

rs (k

t)

AAE

RMSE


COAMPS-TC Improvement

Track and intensity verification results for the 2013 Western Pacific basin season, for a homogeneous comparison of the control version (used in 2013) and a new version of COAMPS-TC. The intensity results show mean absolute error (solid) and mean error (dashed), while the track results show mean absolute error.


Ensembles

Ensemble averages can reduce mean error

2013 HWRF Experimental ensemble track and intensity forecast performance for the NATL basin. Red lines show the operational HWRF and blue the 20-member HWRF ensemble mean (HWMN). The ensemble model was identical to the operational HWRF model run at 3km resolution.


Consensus

Equal weights Unequal weights (based on performance) User (human) selection Examples:

IVCN – NHC, equal weights, n>1ICON – NHC, equal weights, n=4FSSE – FSU, unequal weights, n>1S5XX – JTWC, equal weights, n>1S5YY – JTWC, equal weights, n>1


Consensus Improvements over ST5D

More skillful models -> more independence -> more consensus skill


Unequal or Equal Weights?

The weighted (FSSE) didn’t improve over equal weights (IVCN). There is work involved (bias correction and weight derivation) to do the weights.


From Economics & Finance

“….combination methods have gained even more ground in the forecasting literature, largely because of the strength of the empirical evidence suggesting that these methods systematically perform better than alternatives based on forecasts from a single model. Stable, equal weights have so far been the workhorse of the combination literature and have set a benchmark that has proved surprisingly difficult to beat. This is surprising since−on theoretical grounds−one would not expect any particular combination scheme to be dominant, since the various methods incorporate restrictions on the covariance matrix that are designed to trade off bias against reduced parameter estimation error. The optimal bias can be expected to vary across applications, and the scheme that provides the best trade-off is expected to depend on the sample size, the number of forecasting models involved, the ratio of the variance of individual models’ forecast errors as well as their correlations and the degree of instability in the underlying data generating process.” - Timmermann (2005)

"At least since the publication of “The Combination of Forecasts” (Bates and Granger [1969]), economists have known that combining forecasts from different sources can both improve accuracy and reduce forecaster error. In the intervening years, numerous studies have confirmed these conclusions, outlined conditions under which forecast combinations are most effective, and tried to explain why simple equal weights work so well relative to more sophisticated statistical techniques.” – Jones (2014)


Probabilistic Guidance

Ensemble spread (NWP models)RI probabilities (SHIPS-based, on text output)WANI (analog-based)GPCE (consensus-based)


Guidance Issues?

Carl: “I'd keep playing. I don't think the heavy stuff's gonna come down for quite awhile.”

Caddy Shack (1980)


Guidance Issues?

Carl: “I'd keep playing. I don't think the heavy stuff's gonna come down for quite awhile.”

A forecast bust …

Caddy Shack (1980)


Forecast Issues: Genesis/Formation

Hurricane strike probability from EPS (European Centre Ensemble) 29 March 2014 at 12 UTC valid for the 48 hours period from 30 March at 12 UTC to 01 April at 12 UTC. The hurricane probability was still at zero on 29 March at 12 UTC (TC HELLEN was then a tropical storm), while TC HELLEN became a very intense cyclone 24 hours later (top right = Metop 2, 29 March at 0635 UTC; bottom right = Noaa 19, 30 March at 1042 UTC).


Forecast Issues: Rapid Intensification

Selected intensity guidance (rainbow colors) initiated on August 1, 2014 at 12 UTC and verifying best track intensity (black typhoon symbols) for Halong (11W). Note that all the forecasts are well below the verifying intensity between 24 and 48 h.


Verification

Courtesy: treeofwishes.com


Verification Issues

Does MAE capture everything we want in verification? See Aberson (2008) for some issues and solutions.


Conclusions

The guidance has improved

Agency procedures

Basic process the same

Plenty of subjectivity

Deterministic guidance appears to be of benefit

NWP models are now competitive with other techniques

Consensus reduces MAE

Forecast busts (e.g., genesis and RI) are still a problem

Use performance metrics with caution


Recommendations

Increase sharing of data among forecast centers. More guidance has been developed since the last IWTC and sharing this guidance among the forecast agencies benefits all centers.

Continue development of NWP models, statistical models and probabilistic guidance with special focus on cases with large forecast errors.

Continue to increase the availability of TC intensity guidance by making methods developed in the eastern North Pacific and Atlantic available in other basins.

Investigate new and innovative verification metrics and methods to aid in the elimination of the occasional very poor intensity

forecasts. Encourage research activity to investigate and understand the

physical causes and the impact observation quality/availability on very poor intensity forecasts.



Some Definitions SFMR – Special frequency mircowave imager

A/C – Aircraft

Scatt – Scatterometer

V – Intensity

STIPS – Statistical typhoon intensity prediction system

SHIPS – Statistical hurricane intensity prediction system

SCIP – Statistical cyclone intensity prediction system

GPCE – Goerss predicted consensus error

WANI – Analog model developed by Naval Postgraduate School

IVCN – Intensity variable consensus

ICON – Intensity consensus

FSSE – Florida State Superensemble

S5XX – Intensity consensus at JTWC, uses STIPS

S5YY – Intensity consensus at JTWC, uses SHIPS and H-WRF

RI – Rapid intensification

CLIPER – Climatology and persistence

SHF5 – CLIPER model for intensity in the Atlantic and eastern North Pacific

ST5D – CLIPER model for intensity in the western North Pacific, Indian Ocean and Southern Hemisphere

GFDL – NWP model developed by the GFDL

H-WRF – Hurricane WRF

AROME – NWP model run by METEO-France

COAMPS-TC – Tropical cyclone model developed by U.S. Navy

SCORPIO – IMD web page for satellite imagery and model runs

Documents

2.3.3 ADVANCES IN INTENSITY GUIDANCE Rapporteurs: Buck Sampson (NRL – USA) and John Knaff (NESDIS – USA) Working group: Joe Courtney (BOM - AUS), Brian