Verification of the Cooperative Institute for Precipitation Systems‘
Analog Guidance Probabilistic Products
Chad M. Gravelle and Dr. Charles E. GravesSaint Louis University - Department of Earth and Atmospheric Sciences
John P. GaganNOAA/NWS Springfield, MO
Fred H. GlassNOAA/NWS St. Louis, MO
Michael S. EvansNOAA/NWS Binghamton, NY
2009 National Weather Association Annual Meeting - Norfolk, Virginia20 October 2009
What:• If the current state of the atmosphere resembles a previous
state then the two are termed analogs, and for a period of time, the current state may evolve in a similar fashion as the past state (Lorenz 1969).
• Recent research has had success using the “perfect prog” approach to find analogs.– Hanson (2007)– Root et al. (2007)– Diomede et al. (2008)– Evans and Murphy (2008)
Why:• Provides a conditional climatology of the forecast• Confidence in NWP model output• Historical Impacts - NCDC Storm Data, COOP snow event maps,
snowfall potential, etc.• Historical framework
The What and Why of Analogs
CIPS Analog Guidance - The Big Picture
• Search the 29-yr North American Regional Reanalysis (NARR) dataset against the model forecast (GFS212-40km) for potential analogs.– 6 months over the winter season (OCT - MAR)– 6 h temporal resolution– ~21,112 potential analogs (29 winters, 6 months, 4 per day)
• Remove “duplicate” times by choosing the “best” analog over a 24-h period. 1984011512, 1984011518, 1984011600, 1984011606
• Refine and rank the resulting analogs.
• Create products that are useful for winter weather guidance.
300 HGHT500 HGHT700 FRNT850 HGHT850 TMPC850 TMPC
850 FRNT850
THTEADV
2m TMPCPMSLPWTR
CIPS Analog Guidance Products - Examples
CIPS Analog Guidance Products - Examples
CIPS Analog Guidance Products - Examples
CIPS Analog Guidance Products - Examples
CIPS Analog Guidance Products - Examples
CIPS Analog Guidance Products - Examples
• Establish a basic set of metrics to assess future improvements in the analog system.
• Do the top analogs consistently identify snowfall potential (e.g., does the >6” 50% probability guidance capture the area of >6” of snowfall?)?
• Assess the significance of the probabilistic snowfall guidance (e.g., what does a >4” 30% probability of snow indicate?).
• What thresholds and probabilities are the most reliable in providing snowfall guidance?
Goals of the Verification Study
Verification statistics were determined using the following:
• 35 organized (>2”) snow events occurred during the winter of 2008-2009 east of the Rocky Mountains.– 26 events occurred in one of the Midwest domains.– 14 events occurred in one of the East Coast domains.– 5 events occurred in both Midwest and East Coast domains.
• Four GFS forecasts (F036, F048, F060, F072) and the run’s associated CIPS Analog Guidance Probabilistic Products were used for each event and domain where applicable; 180 total forecast runs.
• Seasonal verification statistics represent the average over the 180 forecast runs.
Winter 08-09 Organized Snow Event Verification Data
• Winter storm affected the CONUS from the mid-Mississippi River Valley through New England between 27 and 29 January 2009.
• Continental United States mid-latitude cyclone track.
East Coast Domain 1 Case - 27-29 January 2009
• Area of >8” snow fell from western New York into interior New England with amounts >12” in higher elevations.
• A mix of snow, sleet, and freezing rain fell to the south of the heavy snow.
• Analogous to the winter storm that affected the Northeast from 15-17 December 2007 (inset). Similar track, snowfall amounts, and precipitation type. COOP Snow Event for the 72-h period ending 1200 UTC
20090129
3rd Best Analog
East Coast Domain 1 Case - >4” Probability Guidance
East Coast Domain 1 Case - >4” Probability Guidance
East Coast Domain 1 Case - >4” Probability Guidance
East Coast Domain 1 Case - >4” Probability Guidance
4360 1319
66714844
POD: 0.867 (1.0)FAR: 0.232 (0.0)THREAT SCORE: 0.687
(1.0)BIAS: 1.13 (1.0)
ObservedYes No
Fore
cast Yes
No
>40% probability:
Winter 08-09 Probabilistic Snow Guidance Results
• The attribute diagram plots the observed relative frequency against the forecast probability (in our case the analog guidance).
• How well do the predicted probabilities of an event correspond to their observed frequencies?
• Reliability is indicated by the proximity of the plotted curve to the diagonal. How close are you to perfect reliability?
• Resolution indicates the ability to assess the change in frequency. How close is your curve to a 1:1 slope ?
Attribute Diagrams
WWRP/WGNE Forecast Verification Research
• The attribute diagram plots the observed relative frequency against the forecast probability (in our case the analog guidance).
• How well do the predicted probabilities of an event correspond to their observed frequencies?
• Reliability is indicated by the proximity of the plotted curve to the diagonal. How close are you to perfect reliability?
• Resolution indicates the ability to assess the change in frequency. How close is your curve to a 1:1 slope ?
Attribute Diagrams
WWRP/WGNE Forecast Verification Research
East Coast Domain 1 Case - >4” Probability Guidance
• Contour area shows the region where analog probability guidance is 40%.
• Observed Relative Frequency: 0.395
• Example of nearly perfect reliability.
East Coast Domain 1 Case - >4” Probability Guidance
• Contour area shows the region where analog probability guidance is 50%.
• Observed Relative Frequency: 0.678
• Example of under forecasting.
East Coast Domain 1 Case - >4” Probability Guidance
• Contour area shows the region where analog probability guidance is 30%.
• Observed Relative Frequency: 0.214
• Example of over forecasting.
Winter 08-09 Probabilistic Snow Guidance Results
• POD results indicate at a given snowfall threshold that the area enclosed by the lower percentiles (10-40%) contain the majority of the threshold snowfall.
• The threat scores indicate that for all thresholds there is a maximum in the 30-50% percentile range.
• The attribute diagrams reveal an expected “overforecasting” of guidance probabilities at all thresholds. However, results between 30-70% have the most reliability.
• In all thresholds, higher guidance probabilities indicate higher relative observed frequency of snowfall (i.e., good resolution).
• Though not examined, we speculate that higher guidance probabilities at low thresholds (i.e., 80% at 4”) may be a better indicator of higher end snowfall potential than low probabilities at high thresholds (i.e., 30% at 8”).
Conclusions
NWA 2009 Analog Posters:
P3.19 Using Regional Historical Analogs as Guidance for a Midwestern Winter Weather Event
P3.22 Use of a Historical Analog-Based Winter Storm Guidance Package for Forecasting a Central New York Snow Event
Questions or comments? [email protected] or [email protected]
The analog guidance can be found at:http://www.eas.slu.edu/CIPS/ANALOG/analog.php
Questions