Verification of the Cooperative Institute for Precipitation Systems‘ Analog Guidance Probabilistic...

Verification of the Cooperative Institute for Precipitation Systems‘

Analog Guidance Probabilistic Products

Chad M. Gravelle and Dr. Charles E. GravesSaint Louis University - Department of Earth and Atmospheric Sciences

John P. GaganNOAA/NWS Springfield, MO

Fred H. GlassNOAA/NWS St. Louis, MO

Michael S. EvansNOAA/NWS Binghamton, NY

2009 National Weather Association Annual Meeting - Norfolk, Virginia20 October 2009

What:• If the current state of the atmosphere resembles a previous

state then the two are termed analogs, and for a period of time, the current state may evolve in a similar fashion as the past state (Lorenz 1969).

• Recent research has had success using the “perfect prog” approach to find analogs.– Hanson (2007)– Root et al. (2007)– Diomede et al. (2008)– Evans and Murphy (2008)

Why:• Provides a conditional climatology of the forecast• Confidence in NWP model output• Historical Impacts - NCDC Storm Data, COOP snow event maps,

snowfall potential, etc.• Historical framework

The What and Why of Analogs

CIPS Analog Guidance - The Big Picture

• Search the 29-yr North American Regional Reanalysis (NARR) dataset against the model forecast (GFS212-40km) for potential analogs.– 6 months over the winter season (OCT - MAR)– 6 h temporal resolution– ~21,112 potential analogs (29 winters, 6 months, 4 per day)

• Remove “duplicate” times by choosing the “best” analog over a 24-h period. 1984011512, 1984011518, 1984011600, 1984011606

• Refine and rank the resulting analogs.

• Create products that are useful for winter weather guidance.

300 HGHT500 HGHT700 FRNT850 HGHT850 TMPC850 TMPC

850 FRNT850

THTEADV

2m TMPCPMSLPWTR

CIPS Analog Guidance Products - Examples

CIPS Analog Guidance Products - Examples

CIPS Analog Guidance Products - Examples

CIPS Analog Guidance Products - Examples

CIPS Analog Guidance Products - Examples

CIPS Analog Guidance Products - Examples

• Establish a basic set of metrics to assess future improvements in the analog system.

• Do the top analogs consistently identify snowfall potential (e.g., does the >6” 50% probability guidance capture the area of >6” of snowfall?)?

• Assess the significance of the probabilistic snowfall guidance (e.g., what does a >4” 30% probability of snow indicate?).

• What thresholds and probabilities are the most reliable in providing snowfall guidance?

Goals of the Verification Study

Verification statistics were determined using the following:

• 35 organized (>2”) snow events occurred during the winter of 2008-2009 east of the Rocky Mountains.– 26 events occurred in one of the Midwest domains.– 14 events occurred in one of the East Coast domains.– 5 events occurred in both Midwest and East Coast domains.

• Four GFS forecasts (F036, F048, F060, F072) and the run’s associated CIPS Analog Guidance Probabilistic Products were used for each event and domain where applicable; 180 total forecast runs.

• Seasonal verification statistics represent the average over the 180 forecast runs.

Winter 08-09 Organized Snow Event Verification Data

• Winter storm affected the CONUS from the mid-Mississippi River Valley through New England between 27 and 29 January 2009.

• Continental United States mid-latitude cyclone track.

East Coast Domain 1 Case - 27-29 January 2009

• Area of >8” snow fell from western New York into interior New England with amounts >12” in higher elevations.

• A mix of snow, sleet, and freezing rain fell to the south of the heavy snow.

• Analogous to the winter storm that affected the Northeast from 15-17 December 2007 (inset). Similar track, snowfall amounts, and precipitation type. COOP Snow Event for the 72-h period ending 1200 UTC

20090129

3rd Best Analog

East Coast Domain 1 Case - >4” Probability Guidance

East Coast Domain 1 Case - >4” Probability Guidance

East Coast Domain 1 Case - >4” Probability Guidance

East Coast Domain 1 Case - >4” Probability Guidance

4360 1319

66714844

POD: 0.867 (1.0)FAR: 0.232 (0.0)THREAT SCORE: 0.687

(1.0)BIAS: 1.13 (1.0)

ObservedYes No

Fore

cast

Yes

No

>40% probability:

Winter 08-09 Probabilistic Snow Guidance Results

• The attribute diagram plots the observed relative frequency against the forecast probability (in our case the analog guidance).

• How well do the predicted probabilities of an event correspond to their observed frequencies?

• Reliability is indicated by the proximity of the plotted curve to the diagonal. How close are you to perfect reliability?

• Resolution indicates the ability to assess the change in frequency. How close is your curve to a 1:1 slope ?

Attribute Diagrams

WWRP/WGNE Forecast Verification Research

• The attribute diagram plots the observed relative frequency against the forecast probability (in our case the analog guidance).

• How well do the predicted probabilities of an event correspond to their observed frequencies?

• Reliability is indicated by the proximity of the plotted curve to the diagonal. How close are you to perfect reliability?

• Resolution indicates the ability to assess the change in frequency. How close is your curve to a 1:1 slope ?

Attribute Diagrams

WWRP/WGNE Forecast Verification Research

East Coast Domain 1 Case - >4” Probability Guidance

• Contour area shows the region where analog probability guidance is 40%.

• Observed Relative Frequency: 0.395

• Example of nearly perfect reliability.

East Coast Domain 1 Case - >4” Probability Guidance

• Contour area shows the region where analog probability guidance is 50%.

• Observed Relative Frequency: 0.678

• Example of under forecasting.

East Coast Domain 1 Case - >4” Probability Guidance

• Contour area shows the region where analog probability guidance is 30%.

• Observed Relative Frequency: 0.214

• Example of over forecasting.

Winter 08-09 Probabilistic Snow Guidance Results

• POD results indicate at a given snowfall threshold that the area enclosed by the lower percentiles (10-40%) contain the majority of the threshold snowfall.

• The threat scores indicate that for all thresholds there is a maximum in the 30-50% percentile range.

• The attribute diagrams reveal an expected “overforecasting” of guidance probabilities at all thresholds. However, results between 30-70% have the most reliability.

• In all thresholds, higher guidance probabilities indicate higher relative observed frequency of snowfall (i.e., good resolution).

• Though not examined, we speculate that higher guidance probabilities at low thresholds (i.e., 80% at 4”) may be a better indicator of higher end snowfall potential than low probabilities at high thresholds (i.e., 30% at 8”).

Conclusions

NWA 2009 Analog Posters:

P3.19 Using Regional Historical Analogs as Guidance for a Midwestern Winter Weather Event

P3.22 Use of a Historical Analog-Based Winter Storm Guidance Package for Forecasting a Central New York Snow Event

Questions or comments? gravelle@eas.slu.edu or gravesce@slu.edu

The analog guidance can be found at:http://www.eas.slu.edu/CIPS/ANALOG/analog.php

Questions