Upload
marcel
View
27
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Skill Variability Diagnosis for Discriminating Use of CPC Long-Lead Seasonal Forecasts. Bob Livezey and Marina Timofeyeva NOAA/NWS/OCWWS/Climate Services Division. Climate Prediction Applications Science Workshop March 9-11, 2004 Tallahassee, FL. Outline. Introduction - PowerPoint PPT Presentation
Citation preview
Skill Variability Diagnosis for Discriminating Skill Variability Diagnosis for Discriminating Use of CPC Long-Lead Seasonal ForecastsUse of CPC Long-Lead Seasonal Forecasts
Bob Livezey and Marina TimofeyevaNOAA/NWS/OCWWS/Climate Services Division
Climate Prediction Applications Science WorkshopMarch 9-11, 2004Tallahassee, FL
OutlineOutline
• IntroductionIntroduction• CPC Skill Graphs and Some Other CPC Skill Graphs and Some Other
StratificationsStratifications• ResultsResults• Conclusions and LessonsConclusions and Lessons
IntroductionIntroduction
• Users should only care about the performance of forecasts that can potentially benefit their decision process
• Livezey (1990): There are non-random subsets of seasonal forecasts that were skillful enough to be useful
• After 1997-98 a common presumption was
that forecasts are generally skillful enough to be useful
Seasonal Temperature Forecast Skill Seasonal Temperature Forecast Skill 1960s to 80s1960s to 80s
All Seasons 8.3
Winter 12.6Spring 8.6Summer 9.3Fall 2.8
Introduction (Cont.)Introduction (Cont.)
• This talk will make the point :
– That was made by Livezey (1990)– That there are many non-random subsets of
forecasts that do not have useful skill – That it is critical for this information to be shared
with potential users– That skill analyses with different stratifications are
highly informative while CPC’s web displays are not
Displays and StratificationsDisplays and Stratifications
• CPC Seasonal Forecasts– For 3-equally probable temperature and precipitation classes at 102
Climate Divisions– Made every month from 1995 to present for 0.5-, 1.5-, …, 12.5 month
leads
• Skill Measure: Modified Heidke Skill Score of Categorized Forecasts
• Displays and Stratifications– CPC: Summed over all forecasts for each lead and displayed with
times series for this lead– Here:
• Summed over all forecasts for each lead and all leads displayed together• Stratified further by cold seasons (DJF to FMA) and warm seasons (MAM
to NDJ)• Stratified further by strong ENSO years vs. other years• Stratified by region
Displays and StratificationsDisplays and Stratifications
• CPC Seasonal Forecasts– For 3-equally probable temperature and precipitation classes at 102
Climate Divisions– Made every month from 1995 to present for 0.5-, 1.5-, …, 12.5 month
leads
• Skill Measure: Modified Heidke Skill Score of Categorized Forecasts
• Displays and Stratifications– CPC: Summed over all forecasts for each lead and displayed with
times series for this lead– Here:
• Summed over all forecasts for each lead and all leads displayed together• Stratified further by cold seasons (DJF to FMA) and warm seasons (MAM
to NDJ)• Stratified further by strong ENSO years vs. other years• Stratified by region
Stratification by Lead and Stratification by Lead and Seasons: TemperatureSeasons: Temperature
Heidke Skill Scores for All Years
-10
0
10
20
30
40
50
0.5 1.5 2.5 3.5 4.5 5.5 6.5 7.5 8.5 9.5 10.5 11.5 12.5
Lead (month)
Hei
dke
SS
.
All Seasons Cold Seasons WarmSeasons
Further Stratification by Strong-Further Stratification by Strong-ENSO vs Other Years: Temp.ENSO vs Other Years: Temp.
Heidke Scores for Cold Seasons(DJF, JFM, FMA)
-10
0
10
20
30
40
50
0.5 1.5 2.5 3.5 4.5 5.5 6.5 7.5 8.5 9.5 10.5 11.5 12.5
Lead (month)
Hei
dke
SS
.
All Years 1997-2000 Other Years
Further Stratification by Strong-Further Stratification by Strong-ENSO vs Other Years: Temp.ENSO vs Other Years: Temp.
Heidke Scores for Warm Seasons(MAM - NDJ)
-10
0
10
20
30
40
50
0.5 1.5 2.5 3.5 4.5 5.5 6.5 7.5 8.5 9.5 10.5 11.5 12.5
Lead (month)
Hei
dke
SS
.
All Years 1997-2000 Other Years
Displays and StratificationsDisplays and Stratifications
• CPC Seasonal Forecasts– For 3-equally probable temperature and precipitation classes at 102
Climate Divisions– Made every month from 1995 to present for 0.5-, 1.5-, …, 12.5 month
leads
• Skill Measure: Modified Heidke Skill Score of Categorized Forecasts
• Displays and Stratifications– CPC: Summed over all forecasts for each lead and displayed with
times series for this lead– Here:
• Summed over all forecasts for each lead and all leads displayed together• Stratified further by cold seasons (DJF to FMA) and warm seasons (MAM
to NDJ)• Stratified further by strong ENSO years vs. other years• Stratified by region
Stratification by Lead and Stratification by Lead and Regions: Temp.Regions: Temp.
Heidke Skill Scores for All Seasons / All Years
0
5
10
15
20
0.5 1.5 2.5 3.5 4.5 5.5 6.5 7.5 8.5 9.5 10.5 11.5 12.5
Lead (month)
Heid
ke S
S .
Western States Eastern States Allstates
ResultsResults
• Seasonal Temperature:– Useable skill confined to strong ENSO years and mainly
at short to medium leads– Otherwise skill is dominantly level with lead (derived
from biased climatologies, ie long-term trend)– Best forecasts are for strong-ENSO cold seasons at
very short leads– Worst forecasts are for cold seasons at longer leads for
strong ENSOs and at very-short leads for other years– Skill is substantially higher than average in the West,
and substantially lower in the East– Short-lead forecasts are better now than for the 1960s-
80s, ~13 vs ~8 overall, ~20 vs ~13 for the winter
Stratification by Lead and Seasons: Stratification by Lead and Seasons: PrecipitationPrecipitation
Heidke Skill Scores for All Years
-10
-5
0
5
10
0.5 1.5 2.5 3.5 4.5 5.5 6.5 7.5 8.5 9.5 10.5 11.5 12.5
Lead (month)
Hei
dke
SS
.
All Seasons Cold Seasons WarmSeasons
Further Stratification by Strong-Further Stratification by Strong-ENSO vs Other Years: Precip.ENSO vs Other Years: Precip.
Heidke Scores for Cold Seasons(DJF, JFM, FMA)
-10
-5
0
5
10
15
20
0.5 1.5 2.5 3.5 4.5 5.5 6.5 7.5 8.5 9.5 10.5 11.5 12.5
Lead (month)
Hei
dke
SS
.
All Years 1997-2000 Other Years
Further Stratification by Strong-Further Stratification by Strong-ENSO vs Other Years: Precip.ENSO vs Other Years: Precip.
Heidke Scores for Warm Seasons(MAM - NDJ)
-10
-5
0
5
10
15
20
0.5 1.5 2.5 3.5 4.5 5.5 6.5 7.5 8.5 9.5 10.5 11.5 12.5
Lead (month)
Hei
dke
SS
All Years 1997-2000 Other Years
Stratification by Lead and Stratification by Lead and Regions: Precip.Regions: Precip.
Heidke Skill Scores for All Seasons / All Years
-2
-1
0
1
2
3
4
0.5 1.5 2.5 3.5 4.5 5.5 6.5 7.5 8.5 9.5 10.5 11.5 12.5
Lead (month)
Heid
ke S
S .
Northern States Southern States Allstates
ResultsResults
• Seasonal Precipitation:
– Barely useable skill entirely confined to strong ENSO years in short to medium leads
– Otherwise skill is either negative or statistically indistinguishable from zero
– Best forecasts are for strong-ENSO cold seasons at short leads
– Skill is a little higher than average in the South, and a little lower in the North
– Short-lead forecasts overall seem to be no better now than for the 1960s-80s
Conclusions and LessonsConclusions and Lessons
• There are non-random subsets of seasonal forecasts that are skillful enough to be useful
• There are many non-random subsets of forecasts that do not have useful skill (should we be issuing them?)
• It is critical for this information to be shared with potential users
• Skill analyses with different stratifications are highly informative while CPC’s web displays are not
• Skills mainly reflect ENSO and trend signals• Temperature forecasts are getting better