A Preliminary Verification of the A Preliminary Verification of the National Hurricane Center’s Tropical National Hurricane Center’s Tropical Cyclone Wind Probability Forecast Cyclone Wind Probability Forecast

ProductProduct

Jackie ShaferJackie ShaferScitor CorporationScitor Corporation

Florida Institute of Technology – Department of Marine and Florida Institute of Technology – Department of Marine and Environmental SystemsEnvironmental Systems

March 6, 2008March 6, 2008

22

OverviewOverview

IntroductionIntroduction MethodologyMethodology ResultsResults ConclusionsConclusions QuestionsQuestions

33

IntroductionIntroduction

Evaluate performance of Tropical Evaluate performance of Tropical Cyclone Wind Probability Forecast Cyclone Wind Probability Forecast ProductProduct

– Issued by National Hurricane CenterIssued by National Hurricane Center

– Related Study: Dr. John Knaff and Related Study: Dr. John Knaff and Dr. Mark DeMariaDr. Mark DeMaria

– Entire Atlantic Basin for 2006 Hurricane SeasonEntire Atlantic Basin for 2006 Hurricane Season

– This Project:This Project:– 2004 – 2007 Hurricane Seasons2004 – 2007 Hurricane Seasons– Florida East Coast: Jacksonville - MiamiFlorida East Coast: Jacksonville - Miami

MethodologyMethodology

44

Data CollectionData Collection 2004 Hurricane Season 2004 Hurricane Season

Provided by Dr. Mark DeMariaProvided by Dr. Mark DeMaria

2005 – 2007 Hurricane Seasons 2005 – 2007 Hurricane Seasons Provided by NHCProvided by NHC

55

Methodology cont’d…Methodology cont’d…

Determine if the wind speed criteria “occurred Determine if the wind speed criteria “occurred or “did not occur” during each forecast time or “did not occur” during each forecast time interval for each eventinterval for each event

34Kt

50Kt

64Kt

34Kt

50Kt

64Kt

12 h Forecast Interval: 0600z – 1800z on 09/04/2004

≥34Kt occurred in:• Cocoa Beach• Fort Pierce• West Palm Beach• Miami

≥34Kt did not occur in:• Jacksonville• Daytona Beach

66

Methodology cont’d…Methodology cont’d…

OccurredOccurred Did Not Did Not OccurOccur

Forecast % ≥ Forecast % ≥ ThresholdThreshold HitHit FAFA

Forecast % < Forecast % < ThresholdThreshold MissMiss CNCN

Classification of Probability Classification of Probability ForecastsForecasts

““HitHit”: Event Forecast to Occur, Event Occurred”: Event Forecast to Occur, Event Occurred ““MissMiss”: Event Forecast Not to Occur, Event Occurred”: Event Forecast Not to Occur, Event Occurred ““False AlarmFalse Alarm”: Event Forecast to Occur, Event Did Not Occur”: Event Forecast to Occur, Event Did Not Occur ““Correct NegativeCorrect Negative”: Event Forecast Not to Occur, Event Did Not ”: Event Forecast Not to Occur, Event Did Not

OccurOccur

Table 1: Contingency Table showing classification of each probability forecast

77

Methodology cont’d…Methodology cont’d…

Probability of Detection (POD): Probability of Detection (POD): The The fraction of the observed events that were correctly fraction of the observed events that were correctly forecastforecast

Probability of False Detection (POFD) Probability of False Detection (POFD) “False Alarm Rate”: “False Alarm Rate”: A measure of the A measure of the product’s ability to forecast non-eventsproduct’s ability to forecast non-events

Range: 0 to 1; Perfect Score: 1Range: 0 to 1; Perfect Score: 1

Range: 0 to 1; Perfect Score: 0Range: 0 to 1; Perfect Score: 0

88

Methodology cont’d…Methodology cont’d…

OccurredOccurred Did Not Did Not OccurOccur

Forecast % ≥ Forecast % ≥ ThresholdThreshold HitHit FAFA

Forecast % < Forecast % < ThresholdThreshold MissMiss CNCN

Table 1: Contingency Table showing classification of each probability forecast

Probability of Detection:Probability of Detection:

Probability of False Detection; “False Alarm Rate”:Probability of False Detection; “False Alarm Rate”:

99

100%

8% “Optimal Threshold”0%

1010

Methodology cont’d…Methodology cont’d…

OccurredOccurred Did Not Did Not OccurOccur

Forecast % ≥ Forecast % ≥ ThresholdThreshold HitHit FAFA

Forecast % < Forecast % < ThresholdThreshold MissMiss CNCN

Classification of Probability Classification of Probability ForecastsForecasts

““HitHit”: Event Forecast to Occur, Event Occurred”: Event Forecast to Occur, Event Occurred ““MissMiss”: Event Forecast Not to Occur, Event Occurred”: Event Forecast Not to Occur, Event Occurred ““False AlarmFalse Alarm”: Event Forecast to Occur, Event Did Not Occur”: Event Forecast to Occur, Event Did Not Occur ““Correct NegativeCorrect Negative”: Event Forecast Not to Occur, Event Did Not ”: Event Forecast Not to Occur, Event Did Not

OccurOccur

Table 1: Contingency Table showing classification of each probability forecast

1111

Forecast Interval Threshold 12 h 5% 24 h 10% 36 h 10% 48 h 12% 72 h 9% 96 h 4% 120 h 2%

Forecast Interval Threshold 12 h 2% 24 h 1% 36 h 4% 48 h 9% 72 h 2% 96 h 1% 120 h 1%

Forecast Interval Threshold 12 h 1% 24 h 5% 36 h 10% 48 h 9% 72 h 4% 96 h 3% 120 h 1%

1212

StatisticsStatistics

Probability of DetectionProbability of Detection Probability of False DetectionProbability of False Detection False Alarm RatioFalse Alarm Ratio Threat ScoreThreat Score Bias ScoreBias Score AccuracyAccuracy True Skill StatisticTrue Skill Statistic

1313

RangesRanges

Forecast Interval 34 50 64

(HR) (Kt) (Kt) (Kt)

12 100 100 100

24 99 94 71

36 92 71 47

48 76 46 28

72 48 25 13

96 40 22 11

120 20 10 6

1414

ConclusionsConclusions

The product performs wellThe product performs well Product shows high accuracy (0.98 to 0.66) between

12-Hr and 120-Hr, respectively Product adequately distinguishes observed events

from non-observed events: TSS ranges from 0.97 to 0.25 for the 12-Hr to 120-Hr,

respectively

Probabilities that may seem small or Probabilities that may seem small or unimportant are actually significantunimportant are actually significant

1515

So What?So What?

Results will be useful for:Results will be useful for: Operational decisions at CCAFS and KSCOperational decisions at CCAFS and KSC

Shuttle RollbackShuttle Rollback Payload ProtectionPayload Protection Personnel EvacuationPersonnel Evacuation

Risk Management Risk Management Evaluating Cost-Risk-Benefit Ratios for Evaluating Cost-Risk-Benefit Ratios for

evacuation decisionsevacuation decisions

Additional Areas of Interest:Additional Areas of Interest: Corpus Christi, TXCorpus Christi, TX New Orleans, LANew Orleans, LA Charleston, SCCharleston, SC

Questions?