Topic 1.5: Operational guidance and skill in forecasting structure change Presented by John Knaff CIRA/CSU Working Group:C. Guard, J. Kossin, T. Marchok,

Topic 1.5: Operational Topic 1.5: Operational guidance and skill in guidance and skill in forecasting structure changeforecasting structure change

Presented by Presented by John KnaffJohn KnaffCIRA/CSUCIRA/CSU

Working Group:Working Group: C. Guard, J. Kossin, T. Marchok, B. C. Guard, J. Kossin, T. Marchok, B. Sampson, Sampson,

T. Smith, N. SurgiT. Smith, N. Surgi

20-30 November 200620-30 November 2006San Jose, Costa RicaSan Jose, Costa Rica

SIXTH INTERNATIONAL WORKSHOP ON SIXTH INTERNATIONAL WORKSHOP ON TROPICAL CYCLONESTROPICAL CYCLONES

22

Presentation OutlinePresentation Outline

Review Typical Measures of TC Review Typical Measures of TC structurestructure

Operational TC Structure GuidanceOperational TC Structure Guidance Intensity verificationIntensity verification

– Long-term operationalLong-term operational– Long-term guidanceLong-term guidance

Wind Radii VerificationWind Radii Verification– 2005 Atlantic2005 Atlantic

Summary and RecommendationsSummary and Recommendations



33

Measures of TC StructureMeasures of TC Structure

IntensityIntensity– Minimum Sea Level Pressure (MSLP)Minimum Sea Level Pressure (MSLP)– Maximum Surface Winds (MSW)Maximum Surface Winds (MSW)**

Wind StructureWind Structure– Radii of significant winds/wind speed Radii of significant winds/wind speed

thresholds (e.g., Radii of 34-, 50-, 64-kt thresholds (e.g., Radii of 34-, 50-, 64-kt winds) winds) **

Pressure DistributionPressure Distribution– Outer closed or outer closed & circular isobarOuter closed or outer closed & circular isobar

* Are used for the verification presented here* Are used for the verification presented here



44

Current TC Intensity Current TC Intensity Guidance MethodsGuidance MethodsFew MethodsFew Methods 24-h Dvorak extrapolation – 24-h Dvorak extrapolation –

subjective/statisticalsubjective/statistical Purely statistical models based Purely statistical models based

on historical best tracks on historical best tracks (CLIPER, climatology, analogs)(CLIPER, climatology, analogs)

Statistical-dynamical models Statistical-dynamical models that use NWP forecasts of that use NWP forecasts of environmental conditions to environmental conditions to make statistical forecastsmake statistical forecasts

Numerical Weather Prediction Numerical Weather Prediction (NWP) methods that make (NWP) methods that make deterministic forecasts.deterministic forecasts.

Consensus Methods, Consensus Methods, combination of skillful forecast combination of skillful forecast methods methods

WeaknessesWeaknesses Short-term, subjectiveShort-term, subjective Do not use current synoptic Do not use current synoptic

information, conservative best information, conservative best used for verificationused for verification

Conservative - cannot predict Conservative - cannot predict rapid intensity changes, timing rapid intensity changes, timing at long leads, over intensify at long leads, over intensify weak systems, poorly handle weak systems, poorly handle high latitude decayhigh latitude decay

Spin-up issues, over intensify Spin-up issues, over intensify some systems, poor physical some systems, poor physical initialization, parameterized initialization, parameterized physicsphysics

Only as good as the Only as good as the independent guidance independent guidance



55

Current Operational TC Current Operational TC Wind Structure GuidanceWind Structure Guidance

Fewer methodsFewer methods Climatology (tabular, Climatology (tabular,

equation form as a equation form as a function of intensity)function of intensity)– Vary among forecast Vary among forecast

centerscenters Simple Statistical Simple Statistical

(CLIPER models)(CLIPER models)– Basin dependentBasin dependent

Numerical Weather Numerical Weather Prediction.Prediction.

WeaknessesWeaknesses Only as good as the Only as good as the

climatology, not climatology, not documenteddocumented

Based on past Based on past operational estimates operational estimates (are they good?)(are they good?)– Scatterometry, aircraft?Scatterometry, aircraft?

Effected by resolution, Effected by resolution, and vortex initializationand vortex initialization



66

Operational Intensity Operational Intensity VerificationVerification

Caveats:Caveats: Datasets come from the RSMC, Miami Datasets come from the RSMC, Miami

(NOAA/TPC) and USA/Joint Typhoon (NOAA/TPC) and USA/Joint Typhoon Warning Center.Warning Center.

– 1-minute sustained 10-m wind1-minute sustained 10-m wind– Post season re-analyzed intensities making use of Post season re-analyzed intensities making use of

all available intensity estimates (i.e., Dvorak, all available intensity estimates (i.e., Dvorak, AMSU, Quickscat, Aircraft (flight-level, MSLP) etc.)AMSU, Quickscat, Aircraft (flight-level, MSLP) etc.)

Measurements for VerificationMeasurements for Verification– All intensitiesAll intensities1.1. Mean Absolute Errors (MAE)Mean Absolute Errors (MAE)2.2. Percent Reduction in Variance (PRV)Percent Reduction in Variance (PRV)



77

Long-Term Operational Long-Term Operational Intensity Verification (MAE)Intensity Verification (MAE)

N. Atlantic

5

10

15

20

25

30

1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005

Year

Mea

n A

bso

lute

Err

or

(kt)

24

48

72

E. Pacific

5

10

15

20

25

30

1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005

Year

Mea

n A

bso

lute

Err

or

(kt)

24

48

72

All intensities

Forecasts fromRSMC, Miami(NOAA/TPC)



88

Long-Term Operational Long-Term Operational Intensity Verification (MAE)Intensity Verification (MAE)

Southern Hemisphere

5

10

15

20

25

30

1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005

Year

Mea

n A

bso

lute

Err

or

(kt)

24

48

W. Pacific

5

10

15

20

25

30

1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005

Year

Mea

n A

bso

lute

Err

or

(kt)

24

48

72

All intensities

Forecasts fromUSA/JTWC



99

Intensity Verification other Intensity Verification other RSMCsRSMCs

Range 0h 12h 24h 48h 72h

Average error (kt)

3 6 9 14 16

RMSE (kt) 4 6 9 12 13

Bias (kt) -1 -1 -2 -4 -2

Skill against persistence

6% 31% 43% 50%

Sample (number of forecasts verified)

310

303

291

255

213

24-h 48-h 72-h

2004 5.1 (9.9) 7.1 (13.8) 8.1 (15.8)

2003 4.9 (9.5) 6.5 (12.7) 7.6 (14.8)

2002 5.0 (9.7) 7.0 (13.6) N/A

2001 5.2 (10.1) 6.9 (13.4) N/A

2000 5.9 (11.5) N/A N/A

Tropical Storm Intensity

2004-2005RSMC, La Reunion

2000-2004RSMC, Tokyom/s & (kt)

Taken from annual reports



1010

Trends in MAETrends in MAE

24-h 48-h 72-h

ATL (1986-2005)

-0.8 -1.0 -1.4

EPAC (1990-2005)

0.0 -0.8 -1.9

SHEM (1991-2005)

2.0 2.8 N/A

WPAC (1986-2005)

-0.2 -0.6 -1.1

kt per decade

Significance is marginal (70%) using annual number of degrees of freedom



1111

Long-Term Operational Long-Term Operational Intensity Verification (PRV)Intensity Verification (PRV)

N

n

N

nn

oo

poPRV

1

2

1

2

)(

)(0.1100

Where o is the observed intensity and p is the predicted intensity and the overbar represents a mean value.

• Can be negative if the numerator (forecast error variance) is larger than the denominator (climatological variance)•This methodology penalizes forecasts methodologies for having non-random errors (e.g., bias)

Variance of the forecast errors

Variance climatological errors



1212


24-h Variance Explained

-60%

-40%

-20%

0%

20%

40%

60%

80%

1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005

Year

Per

cen

t R

edu

ctio

n i

n V

aria

nce

WPAC

ATL

EPAC

SHEM


-60%

-40%

-20%

0%

20%

40%

60%

80%

1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005

Year

Per

cen

t R

edu

ctio

n i

n V

aria

nce

WPAC

ATL

EPAC

SHEM

All intensities

Forecasts fromRSMC, Miami(NOAA/TPC) in EP and ATL

All intensities

Forecasts fromUSA/JTWC in WP and SH



1313



-60%

-40%

-20%

0%

20%

40%

60%

80%

1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005

Year

Per

cen

t R

edu

ctio

n i

n V

aria

nce

WPAC

ATL

EPAC



1414

Trends in PRVTrends in PRV

24-h 48-h 72-h

ATL (1986-2005) 14.7 22.3 41.0

EPAC (1990-2005) 1.5 1.4 3.3

SHEM (1991-2005) 2.0 2.8 N/A

WPAC (1986-2005) 3.2 3.0 3.6

Percent per decade

Significant (95%) in the Atlantic, marginally significant (70%) in the other basins when based on annual number of degrees of freedom

Intensity Guidance MethodsIntensity Guidance Methods Atlantic and East Pacific

SHIFOR (1988-present) Statistical Hurricane Intensity FORecast, which uses simple climatology and persistence parameters

SHIPS (1991-present, ATLC) Statistical Hurricane Intensity Prediction Scheme, (1996-present, EPAC) which uses climatology, persistence and real-

time atmospheric and oceanic parametersGFDL (1995-present) Operational version of the Geophysical Fluid Dynamics

Laboratory hurricane modelGFDN (2001-present) GFDL model initialized from Navy global model fieldsSHIFOR5 (2001-present) Updated version of SHIFOR with 5 day forecasts

West PacificCLIM (1985-present) Climatological analog modelSTIFOR (1991-present) Statistical Typhoon FORecast Model, similar to SHIFORGFDN (1995-present) GFDL model initialized from Navy global model fieldsAFW (2000-present) MM5 mesoscale model adapted to typhoon forecasts JTYM (2001-present) Japanese Meteorological Agency limited area typhoon modelST5D (2002-present) Updated STIFOR model and extended to 5 daysSTIPS (2003-present) Statistical Typhoon Intensity Prediction Scheme, similar to

SHIPSST10 (2005-present) Ensemble version of STIPS

Southern HemisphereCLIM(2000- present) Climatological analog.GFDN (2000- present) GFDL model initialized from Navy global model fieldsTC-LAPS (2001-present) BOM limited area prediction systemST5D (2004 –present) 5-day Climatology and persistenceSTIPS(2005 – present) STIPS SH (experimental run at NRLM)ST10 (2005 – present) STIPS SH ensemble (experimental, run at NRLM)



1616

Comparison with Best 48-hr Comparison with Best 48-hr Intensity Guidance Intensity Guidance

Atlantic

-30%

-20%

-10%

0%

10%

20%

30%

40%

50%

60%

70%

80%

1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005

Year

% R

edu

ctio

n i

n V

aria

nce

RMTC, Miami

Guidance

East Pacific

-30%

-20%

-10%

0%

10%

20%

30%

40%

50%

60%

70%

80%

1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005

Year

% R

edu

ctio

n i

n V

aria

nce

RMTC, Miami

Guidance



1717

Comparison with Best 48-hr Comparison with Best 48-hr Intensity GuidanceIntensity Guidance

Southern Hemisphere

-30%

-20%

-10%

0%

10%

20%

30%

40%

50%

60%

70%

80%

1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005

Year

% R

edu

ctio

n i

n V

aria

nce

JTWC

Guidance

West Pacific

-30%

-20%

-10%

0%

10%

20%

30%

40%

50%

60%

70%

80%

1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005

Year

% R

edu

ctio

n i

n V

aria

nce

JTWC

Guidance



1818

Best Guidance (Atlantic, E. Best Guidance (Atlantic, E. Pacific and W. Pacific)Pacific and W. Pacific)

Atlantic East Pacific West Pacific1991 SHIPS SHIFOR CLIM1992 SHIFOR SHIFOR CLIM1993 SHIFOR SHIFOR CLIM1994 SHIFOR SHIFOR CLIM1995 SHIFOR SHIFOR CLIM1996 SHIPS SHIFOR GFDN1997 GFDL SHIPS STIFOR1998 SHIPS SHIPS STIFOR1999 SHIPS SHIPS STIFOR2000 SHIPS SHIFOR STIFOR2001 SHIPS SHIPS ST5D2002 SHIPS GFDL ST5D2003 SHIPS SHIFOR STIPS2004 SHIPS GFDL STIPS2005 SHIPS SHIPS ST102006-preliminary2006-preliminary GFDLGFDL SHIPS/ICONSHIPS/ICON ST10ST10



1919

Summary: Intensity Summary: Intensity ForecastingForecasting There is evidence that forecasts, in general, There is evidence that forecasts, in general,

are improving very slowlyare improving very slowly Operational forecast improvements are being Operational forecast improvements are being

driven by improvements in guidance driven by improvements in guidance methodsmethods

Statistical-dynamical and regional/specialized Statistical-dynamical and regional/specialized NWP guidance are most skillfulNWP guidance are most skillful

Global models do not have skill and have Global models do not have skill and have larger errors than climatology and larger errors than climatology and persistence based forecastspersistence based forecasts

Consensus methods created from skillful Consensus methods created from skillful guidance have been demonstrated to guidance have been demonstrated to produce better forecasts than single methods produce better forecasts than single methods



2020

Wind Radii verificationWind Radii verification

Caveats:Caveats: Post-season reanalyzed estimates of R34 Post-season reanalyzed estimates of R34

used for verificationused for verification– These make use of the best available data These make use of the best available data

(variable)(variable) QuickScat, AMSU, SSMIQuickScat, AMSU, SSMI Flight-level windsFlight-level winds SFMRSFMR Ships/buoysShips/buoys

One year, Atlantic onlyOne year, Atlantic onlyMethods:Methods: MAE in n. miMAE in n. mi Hit & False alarm rates.Hit & False alarm rates.



2121

Wind Radii Wind Radii Guidance/ForecastsGuidance/Forecasts

OFCL – Official RSMC, Miami forecastOFCL – Official RSMC, Miami forecast AVNI – NOAA Global Forecast SystemAVNI – NOAA Global Forecast System GFTI – NOAA/GFDL model forecastsGFTI – NOAA/GFDL model forecasts MRCL – Multiple linear regression MRCL – Multiple linear regression

CLIPER Model, 3-daysCLIPER Model, 3-days DRCL – Statistical-Parametric CLIPER DRCL – Statistical-Parametric CLIPER

model, 5-daysmodel, 5-days



2222

Wind Radii Verification (R-34 Wind Radii Verification (R-34 kt)kt)

10

25

40

55

70

85

100

12-h 24-h 36-h 48-h 72-h

Forecast Hour

MA

E (

n m

i)

OFCL

AVNI

GFDT

MRCL

DRCL

1596 1456 1316 1156 872



2323

Wind Structure VerificationWind Structure Verification

0.2

0.4

0.6

0.8

1

12-h 24-h 36-h 48-h 72-h

Forecast Hour

Hit

Rat

e OFCL

AVNI

GFDT

MRCL

DRCL

0

0.2

0.4

0.6

0.8

1

12-h 24-h 36-h 48-h 72-h

Forecast Hour

Fal

se A

larm

Rat

e

OFCL

AVNI

GFDT

MRCL

DRCL

Probability of DetectionRadii of 34-kt winds

Probability of False DetectionRadii of 34-kt winds

MAE vs. False Alarm Trade off



2424

Influence of Intensity Influence of Intensity Forecast Errors on Wind Forecast Errors on Wind RadiiRadii

0

10

20

30

40

50

60

12-h 24-h 36-h 48-h 72-h 96-h 120-h

Forecast Hour

MA

E (

n m

i)

OFCL

DRCL

DRCC

Forecasts using Best Track Intensities

Forecasts usingForecast intensities

1832 1700 1544 1400 1104 860 668



2525

Influence of Intensity Influence of Intensity Forecast Errors on Wind Forecast Errors on Wind RadiiRadii

0

0.2

0.4

0.6

0.8

1

12-h 24-h 36-h 48-h 72-h 96-h 120-h

Forecast Hour

Fal

se A

larm

Rat

e

OFCLDRCLDRCC

0.5

0.6

0.7

0.8

0.9

1

12-h 24-h 36-h 48-h 72-h 96-h 120-h

Forecast Hour

Hit

Rat

e

OFCL

DRCL

DRCC

False Alarm Decrease

Hit RateIncrease



2626

Summary: Wind Radii Summary: Wind Radii ForecastingForecasting There are very few models There are very few models

– poor understanding of processespoor understanding of processes– poor developmental datasetspoor developmental datasets

There are no skillful modelsThere are no skillful models– No higher level statistical modelsNo higher level statistical models– NWP does not initialize the vortex properly, have NWP does not initialize the vortex properly, have

sufficient resolution (global)sufficient resolution (global) More accurate intensity forecasts will More accurate intensity forecasts will

improve wind radii forecastsimprove wind radii forecasts There is a trade off between false alarm rates There is a trade off between false alarm rates

and MAE in current forecasts schemes (i.e. and MAE in current forecasts schemes (i.e. symmetric forecasts produce smaller MAE symmetric forecasts produce smaller MAE statistics)statistics)



2727

Summary TC Structure Summary TC Structure ForecastingForecastingOverall Forecasting of TC structure change is rather poor.Overall Forecasting of TC structure change is rather poor. The forecasting process is still subjectiveThe forecasting process is still subjective

– Best intensity forecasts are used as a baseline that Best intensity forecasts are used as a baseline that is modified by the forecasteris modified by the forecaster

– Skillful wind radii guidance is unavailableSkillful wind radii guidance is unavailable Intensity forecasts are improving very slowly and are Intensity forecasts are improving very slowly and are

being driven by improved guidancebeing driven by improved guidance Wind radii forecasting is in its infancy and is hindered Wind radii forecasting is in its infancy and is hindered

byby– Poor developmental datasetsPoor developmental datasets– Poor physical understandingPoor physical understanding– Has not been an operational priorityHas not been an operational priority– Only recently have such forecasts been verified. Only recently have such forecasts been verified.



2828

Future/Current Research Future/Current Research and Developmentand Development 1.1. Making better use of existing technology and Making better use of existing technology and

datasets. Examples include :datasets. Examples include :– Diagnostic studies to understand tropical cyclone wind Diagnostic studies to understand tropical cyclone wind

field growthfield growth– Development of models to predict structure change Development of models to predict structure change

using environmental and storm conditionsusing environmental and storm conditions– Model output statistics to predict wind radiiModel output statistics to predict wind radii– Indices to predict rapid intensification, annular Indices to predict rapid intensification, annular

hurricanes, secondary eyewall formation, etc.hurricanes, secondary eyewall formation, etc.– Forecast techniques to improve short-term intensity Forecast techniques to improve short-term intensity

forecasts that leverage existing and longstanding forecasts that leverage existing and longstanding satellite technologiessatellite technologies

– Probabilistic models that account for track, intensity Probabilistic models that account for track, intensity and wind radii error distributionsand wind radii error distributions

– … … Items for discussionItems for discussion



2929

Future/Current Research Future/Current Research and Developmentand Development2.2. Development of new technology.Development of new technology. Next Generation Hurricane Forecast Model, the Next Generation Hurricane Forecast Model, the

Hurricane – Weather Research and Forecasting (H-Hurricane – Weather Research and Forecasting (H-WRF) Model is being tested/developedWRF) Model is being tested/developed

– Physical initialization using a 3D-var framework that Physical initialization using a 3D-var framework that can make use of Dopplar radar information (WR-88D, can make use of Dopplar radar information (WR-88D, airborne) and new instrument packages on the G-IV, airborne) and new instrument packages on the G-IV, and satellite data. and satellite data.

– Coupling with waves via the Wavewatch III wave modelCoupling with waves via the Wavewatch III wave model– Parameterizations developed using the recent results Parameterizations developed using the recent results

of the CBLAST experimentof the CBLAST experiment New instrumentation on operational aircraft New instrumentation on operational aircraft

– like the SFMR.like the SFMR.– Others etc. Discussion?Others etc. Discussion?



3030

RecommendationsRecommendations

Continued development of consensus methods to Continued development of consensus methods to improve intensity forecasts.improve intensity forecasts.

Use of model output statistics, particularly in wind radii Use of model output statistics, particularly in wind radii prediction.prediction.

Greater effort toward high resolution NWP that has Greater effort toward high resolution NWP that has physical initialization and advanced data assimilation physical initialization and advanced data assimilation capabilitiescapabilities

Effort to make the newest technology/instrumentation Effort to make the newest technology/instrumentation and resulting observations available to real-time and resulting observations available to real-time operational centers and tropical cyclone researchers.operational centers and tropical cyclone researchers.

Additional recommendation based on knowledge gained Additional recommendation based on knowledge gained during the process of writing the topic summaryduring the process of writing the topic summary

There is a need for more operational scatterometry There is a need for more operational scatterometry and similar active remote sensing instrumentation to and similar active remote sensing instrumentation to detect tropical cyclone wind fields. None is currently detect tropical cyclone wind fields. None is currently planned.planned.

Documents

Topic 1.5: Operational guidance and skill in forecasting structure change Presented by John Knaff CIRA/CSU Working Group:C. Guard, J. Kossin, T. Marchok,