Upload
elden
View
42
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Zoltan Toth Environmental Modeling Center NOAA/NWS/NCEP Acknowledgements: Louis Uccellini, Steve Lord & Ensemble Team Paul Schultz, DJ Seo, Tom Hamill, Steve Mullen, Julie Demargne http://wwwt.emc.ncep.noaa.gov/gmb/ens/index.html. - PowerPoint PPT Presentation
Citation preview
1
REVIEW OF RECOMMENDATIONS FROM THE 3RD ENSEMBLE USER WORKSHOP (OCT. 2006)
Zoltan Toth
Environmental Modeling CenterNOAA/NWS/NCEP
Acknowledgements:Louis Uccellini, Steve Lord & Ensemble Team
Paul Schultz, DJ Seo, Tom Hamill, Steve Mullen, Julie Demargne
http://wwwt.emc.ncep.noaa.gov/gmb/ens/index.html
FOLLOW NRC REPORT RECOMMENDATIONSAll major recommendations embraced
• Develop roadmap for assessing & communicating forecast uncertainty– Based on science, technology, workforce considerations– Consistent with NOAA’s mission, NWS plans, embraced by “Enterprise”– Define end goal, 5-10 years horizon
• Adopt ensemble-based forecast process– Maximize forecast skill– Unify scientific, software, and technological infrastructure across NOAA
» Weather, water, climate applications
– Suggested in Mar 2007 to develop roadmap by Sept. 2007
MAJOR RECOMMENDATIONS - 1
• Develop roadmap for assessing & communicating forecast uncertainty
• Revise operational requirements to make them probabilistic – Make probabilistic format the primary requirement
• For each forecast application:– Replace single value/categorical format with probabilistic format as primary requirement– Revise/supplement corresponding performance measures (GPRA)
– Essential for orderly transition from traditional to new forecast process• NWS is requirement-driven organization
– Without clear new requirements, process is doomed– Phased implementation schedule in consultation with responsible offices
• Allows orderly transition to new requirements
– Suggested in Mar 2007 to prepare plan by March 2008
MAJOR RECOMMENDATIONS - 2
• Develop roadmap for assessing & communicating forecast uncertainty (.5 yr)• Revise operational requirements to make them probabilistic (1 year)
• Design, develop, and gradually implement new forecast process– Focus on missing scientific, technological, and human components
• Identify self-contained components with– Clear requirements and interfaces between components
– Define basic capabilities achievable in 2-3 years• Limited but consistent with and leading to end goal
– Full capabilities in 5-10 years
• Interface with NOAA THORPEX program– Research/development to improve skill & utility of probabilistic forecasts– Leverage related major NOAA, national, and international efforts
• Integrate with NWS, NOAA, national activities– NOAA-wide regional service plans– NUOPC planning/development
• Provide long-term funding support through PPBES– Overlap with W&W High Impact Event Theme Team
• Form development teams for specific tasks/components following workshp– Identify potential contributors within & outside of NWS & NOAA
MAJOR RECOMMENDATIONS - 3
1. Continue development of expanded forecast process– Focus on adaptive methods applicable for high impact events
• Collection & use of observations (targeted observations)• Data assimilation (case dependent background error estimation)• Numerical modeling (adaptive resolution & high impact modeling)• Ensemble forecasting (case dependent variations in membership & composition)• Decision support systems (flexible user actions depending on forecast probabilities)
2. Bias correction & downscaling methods for ensembles– Estimate/correct lead-time dependent bias in ensemble forecasts (on model
grid)– Generate fine resolution (NDFD grid) uncertainty/ensemble data
• Establish connection between (bias corrected) coarse model vs. fine user grids– Use reanalysis & hind-casts with operational systems as needed
3. Define summary ensemble information to be used to (ST)– Collapse vast amount of ensemble data for inclusion in expanded
NDFD/NDGD• E.g., 10, 50, and 90 percentile of forecast distribution (in place of single value)
– Manually inspect/modify summary ensemble information
SPECIFIC RECOMMENDATIONS – 1-3
ST – Short Term (2-3 yrs); LT – Long Term (5-10 yrs)
4. Contribute to establishment of NOAA-wide environmental data depository– Expand NDFD/NDGD database to include forecast uncertainty (ST)– Develop capability to hold all ensemble trajectories (LT)
• All members, variables, lead times
5. Develop ensemble interrogation, modification, & product generation tools to
– Derive summary information from ensemble (ST)– Manually modify summary ensemble info (ST)– Derive additional statistics from summary info (product generation, ST)– Automatically modify ensemble trajectories based on modified summary info
(LT)– Derive any info from full ensemble data (product generation, LT)
6. Develop telecommunication facilities to access data– Summary info & limited derived products (ST)– All ensemble forecasts & derived products (LT)
SPECIFIC RECOMMENDATIONS – 4-6
ST – Short Term (2-3 yrs); LT – Long Term (5-10 yrs)
7. Develop unified NWS/NOAA probabilistic verification package to (ST)– Assess statistical reliability and resolution for
• Computing official performance measures• Evaluating value added along forecast chain• Assessing value in newly developed vs. operational techniques
8. Develop & implement comprehensive training to– Prepare all participants for their new roles in expanded forecast process, incl.
• Statistical background• Ensemble methods• Best forecast practices in assessing uncertainty• Applications of probabilistic & other uncertainty information
9. Develop outreach program on use & communication of uncertainty– In partnership with entire Weather, Water & Climate Enterprise
• Determine best ways of communicating uncertainty• Compile sample of Decision Support Systems using uncertainty information
– Establish close partnership with public sector users (e.g., emergency, water management)• User feedback on new activities
– Explore how forecast process can be adapted according to user requirements
SPECIFIC RECOMMENDATIONS – 7-9
ST – Short Term (2-3 yrs); LT – Long Term (5-10 yrs)
NEW DEVELOPMENTS SINCE WG RECOMMENDATIONS• NOAA/NWS Forecast Uncertainty Team (NFUSE) formed
– Early 2007– Assess and discuss status, search for solutions
• AMS ACUF Team formed– Enterprise planning activities
• NOAA/NWS Forecast Uncertainty program formed– Outgrowth of NFUSE– Oversight panel established– Budget planning for out-years
• Uncertainty-related activities wide-spread, affect all forecasting
• What is next?– Coordinated
• Work level plans• Related activities
2008 WORKSHOP CONTRIBUTIONS• Agree on long term plan
– What we want in 5-10 yrs – desired end result/framework– Not how we achieve goal
• Multiple avenues can be tested within single framework
• Agree on what we can achieve in next 2-3 years with expected resources– Limited version of long term goal – interim result/framework– Development of short term solution will contribute to long term plan
• Contribute to forming of Work Teams addressing well defined sub-problems– Separate pieces that can be worked on independently– Leads/principles provide overall coordination
DISCUSS / AGREE ON LONG TERM GOALCapability to answer any question related to future weather, climate, water,
including forecast uncertainty• Considering state of related science, what do we need for that?
– Bias corrected, downscaled ensemble forecasts – Ultimate dataset• Allows derivation of single/joint probabilities
– Information on spatial/temporal/cross-variable correlations– Trajectories (scenarios)
– Work independent of particular product/service needs (all questions covered)• Major technical requirements
– Storage for ensemble data• 20-100 times more data compared to current “NDFD”
– Distribution of / access to ensemble data• 20-100 fold increase in bandwidth
– Technical tools to display / manipulate ensemble data• Special AWIPS-2 capabilities
– Forecasters trained to work with ensemble data• 2-step approach warranted
– First implement limited capability• Will provide large portion of total benefit with relatively small investment
DISCUSS / AGREE ON INTERMEDIATE GOALCapability to provide uncertainty/probabilistic info on any weather
element with some approximation• Considering state of related science, what do we need for that?
– Summary info that allows the generation of single-variable cdf/pdf• Bias corrected, downscaled
– Can be generated based on a single or ensemble NWP• Allows derivation of single-variable probabilities
– Including information on spatial/temporal/cross-variable correlations
• Technical requirements– Storage for summary info
• 3-fold increase in NDFD data– Distribution of / access to ensemble data
• 3-fold increase in bandwidth– Technical tools to display / manipulate ensemble data
• Subset of long term toolkit in AWIPS-2– Connect summary stats & cdf/pdf
– Forecasters trained to work with ensemble data• With right choice of summary info, forecasters eased into new paradigm
• Important elementDefinition of basic format for forecast uncertainty
• Advantages of using a common format– Facilitates flow of info between three major elements in forecast process
• Numerical guidance• Human forecasters• External users
– Provides basis for collaboration among various groups• Inter-comparison of different approaches
• Requirements– Can be easily interpreted by users
• Intuitive meaning– Can be manipulated by forecasters
• Related to current forecast modification practices– Can be easily communicated
• Limited data volume– Can be stored in NDFD
• Related to current forecast format– Additional formats can be easily derived
• Based on standard statistical considerations• Choice of exact format not critical
– As long as software available to generate alternate formats
PHASE 1 PLAN
• Background on suggested type of format– Recommended by Predictability WG at WR/NWS SOO/DOH Workshop (Salt
Lake City, May 2003)– Discussed and recommended by Predictability Workshop at Univ. Wisconsin
(Madison, Mar 2004)– Studied in detail and adopted at UKMET Office (2006)
• Suggested format– Beyond mid-point estimate (50%, or mode, or mean?), include
• Two additional fields for each NDFD variable– 10 percentile value of forecast distribution– 90 percentile value of forecast distribution
• Readiness– Numerical guidance
• Can be generated based on ensemble data, using NAWIPS software– Forecasters
• Can use same tools to manipulate 2 new fields as with most likely field– Users
• UKMET research indicates this is intuitive
CHOICE OF FORMAT FOR ASSESSING AND COMMUNICATING FORECAST UNCERTAINTY
BACK
EXAMPLE FROM UK METOFFICE - TEMP
Courtesy of Mark Roulston
EXAMPLE FROM UK METOFFICE - PRECIP
Courtesy of Mark Roulston
• Does it make sense, reasonable format?– No need to worry (too much)
• Just need a common format to connect three main areas– Alternative formats can/will be generated
• What NDFD grids describe currently?– Most likely value (mode of pdf)?– Expected value (mean of pdf)?– 50% exceedence value (median of pdf)?
• What mid-point parameter an expanded NDFD should have?– What do users need? – What is most intuitive to forecasters?
• Would median (50 percentile) be best choice? Similar to 10, 90 percentile value• What threshold may work best for limits?
– 10 & 90 - not too narrow, not too wide?• 5 & 95 percentile may be too extreme for forecasters?• 20 & 80 may not be inclusive enough?
• What software needs to be developed to generate other formats?– Fit parametric distribution to 3 points
• Roman Krzysztofowicz’ library of 40+ distributions– Any probability or its inverse can be derived for single variables
• Basic probabilistic forecast data-base
QUESTIONS ABOUT SUGGESTED FORMAT
TOOLS NEEDED FOR INTERMEDIATE STEP• Modify summary statistics (forecasters’ role)• Fit parametric distribution to modified summary statistics• Adjust ensemble members to be consistent with official forecast
– For users who need trajectories
• Derive probabilities from fitted cdf/pdf
• Same statistical tools must be used across organization– Ensemble generation, display, manipulation, product generation, etc
• Same tools can be used in final stage to manipulate ensemble distribution– Summary statistics inspected & modified by forecasters– Changes back-propagated to ensemble data set
WORK AREAS• All topics need to be developed in coordination
– Plenary sessions for interactions• Topics A & B are especially connected in each WG
– Same groups discussing both topics in 2 sessions– Third WG session to focus on connections
• Working Group 1– Topic A Operational requirements– Topic B Corporate outreach
• Working Group 2– Topic A Ensemble forecasting– Topic B Statistical post-processing
• Working Group 3– Topic A Ensemble data depository / access – Topic B Database interrogation / forecaster tools
• Working Group 4– Topic A Forecaster’s role– Topic B Forecaster training
WORK AREAS• Operational requirements
– General requirements first to “justify” effort– Detailed requirements later as effort unfolds
• Corporate outreach– Link with ACUF process– Promote quantitative use of uncertainty information in Decision Support
Systems
• Ensemble forecasting– Centerpiece of uncertainty effort– THORPEX
• NOAA, national, international research/development– NAEFS, NUOPC, TIGGE, GIFS
WORK AREAS – CONT• Statistical post-processing
– Problem• Relate coarse resolution biased forecast to user relevant fine resolution
information– Break up problem to facilitate collaboration
• Bias correction of coarse resolution forecast grid wrt NWP analysis– Cheap– Sample of forecasts / hind-casts needed
• Downscaling– Relate coarse resolution NWP and fine resolution observationally based
analyses» Perfect prog approach» No need for hind-casts
• Creation of observationally based fine resolution analysis– Estimate of truth
– Groups can collaborate on each topic• Addressing all 3 problems in one swoop limits collaboration
NAEFS RESULTS
• 8 days total gain in skill
• Downscaling more important than bias correction÷ Less need for
hindcasting?÷ Need for local
observationally based analysis
• Multicenter approach adds 1-2 days skill
NCEP raw-downscaledNCEP biascorrected-downscaled
NCEP raw
NAEFS final
WORK AREAS – CONT 2• Ensemble data depository / access
– Create NOAA digital forecast database (linked with other NOAA data)• Summary statistics in phase 1• All ensemble members in phase 2
– Provide easy access to internal / external users• NOMADS, etc?• Link with multi-center ensembles
– NAEFS – NUOPC – GIFS
• Database interrogation / forecaster tools– Derive information from summary stats / ensembles– Modify summary stats– Back-propagate modified info into ensemble– Verify added value at each step of forecast process
• Need for unified verification capability across forecast process– Common requirement across all WGs
WORK AREAS – CONT 3• Forecaster’s role
– Traditional role• Generate forecast, or modify numerical guidance
– Changing environment• Need to generate very large amount of forecast data• Quality of automatic numerical guidance improves
– New role:• Direct and quality control forecast process
– Identify high impact events» Assign adaptively configurable observational, modeling, etc resources
– Interfere with automated process only during critical high impact events• User outreach
– Interpret probabilistic forecast
• Forecaster training– Prepare for new role related to ensemble / probabilistic forecasting
• Modification• Interpretation• Outreach
GMOS forecast
NAEFS products
Surface Temperature MAE CONUS, Sept. 2007
12Z NDFD vs. 00Z MOS/GMOS/NAEFS
Surface Temperature MAE CONUS, Sept. 2007
00Z GMOS vs. 00Z NAEFS
RTMA Analysis METAR obs. 1221 sites
0.5°C
0.5°C
Surface Temperature Area Mean Bias CONUS, Sept. 2007
12Z NDFD vs. 00Z MOS/GMOS/NAEFS
Surface Temperature Pointwise Bias CONUS, Sept. 2007
00Z GMOS vs. 00Z NAEFS
GMOS forecast
NAEFS products
RTMA Analysis METAR obs. 1221 sites
Pointwise Bias Area Mean Bias
0.6°C
0.6°C
BACKGROUND
Example for using new mask
MDL GMOS & NAEFS Downscaled Forecast Mean Absolute Error w.r.t. RTMA Average For Sept. 2007
12-h GMOSForecast
12-h NAEFS Forecast
For CONUS:NAEFS(1.01) : GMOS(1.59)
36% impr. over GMOS
24-h GMOSForecast
For CONUS:NAEFS(1.45) : GMOS(1.72)
15% impr. over GMOS
MDL GMOS & NAEFS Downscaled Forecast Mean Absolute Error w.r.t. RTMA Average For Sept. 2007
24-h NAEFS Forecast
GMOS forecast
NAEFS products
Surface Temperature MAE CONUS, Sept. 2007
12Z NDFD vs. 00Z MOS/GMOS/NAEFS
Surface Temperature MAE CONUS, Sept. 2007
00Z GMOS vs. 00Z NAEFS
RTMA Analysis METAR obs. 1221 sites
0.5°C
0.5°C
Surface Temperature Area Mean Bias CONUS, Sept. 2007
12Z NDFD vs. 00Z MOS/GMOS/NAEFS
Surface Temperature Pointwise Bias CONUS, Sept. 2007
00Z GMOS vs. 00Z NAEFS
GMOS forecast
NAEFS products
RTMA Analysis METAR obs. 1221 sites
Pointwise Bias Area Mean Bias
0.6°C
0.6°C
Western Rgn Pointwise Bias Western Rgn Area Mean Bias
Western Rgn MAE Western Rgn MAE
Central Rgn Pointwise Bias Central Rgn Area Mean Bias
Central Rgn MAE Central Rgn MAE
Eastern Rgn MAE Eastern Rgn MAE
Eastern Rgn Pointwise Bias Eastern Rgn Area Mean Bias
Southern Rgn MAE Southern Rgn MAE
Southern Rgn Pointwise Bias Southern Rgn Area Mean Bias
NCEP-raw
NAEFS-finalNCEP-raw-downs
NCEP-biasc-downs
BACK Bo et al.
BACKGROUND
Need to clarify goals (be)for(e) brainstorming
• Premise– We accept and plan to follow major NRC recommendations
• What’s needed?– Design, develop, and gradually implement changes in forecast process for
assessing, communicating, and using forecast uncertainty
• Basic capabilities (by 2009-2010)– Detailed draft plan (including low hanging fruit) by Oct 07
• Full capabilities (by 2012-2017)– Conceptual draft plan by Oct 07
– Corporate commitment• Funding for both phases
– Reallocation / reprioritization for phase 1– PPBES for phase 2
• New / revised operational requirements– Part of planning / development
NFUSE GOALS
• Phase 1 (2-3 yrs)– Limited time and resources - Build on existing and emerging efforts
• Not all users within/outside NWS can access ensemble data– Limited capabilities
• Ability to express basic forecasts in probabilistic terms– Uni-variate pdf
» Joint probab. computation assumes probabilities for different variables are independent– Other formats derived from uni-variate pdf
• Phase 2 (5-10 yrs)– Full capabilities
• All interested users within/outside NWS have access to ensemble data– Joint probabilities derived from ensemble
• Exhaustive set of forecast formats/products supported
TWO-PHASE PLAN
BACKGROUND
• Current system– Single value format
• Short-term (2-3 yrs) plan – 3 values format (pdf)– Provide best (bias corrected) numerical guidance in agreed upon format
• Provide bias corrected ensemble data to external and selected internal users– Human forecasters modify numerical guidance using agreed upon format
• Distribute and store agreed upon format– Convert format into single variable pdf format
– External users provided with products in format of their choice• Multiple options
– Software needed to derive user requested format from single value pdf
• Long-term (5-10 yrs) plan – ensemble format– Provide best numerical guidance in agreed upon format– Human forecasters modify numerical guidance using agreed upon format
• Propagate information to modify bias corrected ensemble data– Modified bias corrected ensemble data is complete and final forecast dataset includes
» Uncertainty information regarding spatial, temporal, cross-variable co-variances– Human forecasters need access to ensemble data to assist in
» Manipulation (added value)» Interpretation (user outreach)
– External users provided with products in format of their choice• Multiple options
– Software needed to derive user requested format from ensemble data
LINKS WITH NFUSE PLANS
• Choice / acceptance of format for uncertainty info enables inter-comparisons– Verification using common framework possible
• Added value can be traced
• Choice of THORPEX performance measures– Related to major application areas (previous discussion)
• Use 3-value format to verify uncertainty/probabilistic information?
SYNERGIES
• Plans need to be coordinated between three major areas– Probabilistic numerical guidance for high impact events (THORPEX)
• Provides objective numerical guidance– Human forecasters
• Add value to numerical guidance• Interpret guidance / products
– Products to be distributed to and used by external community• Interface with “Weather enterprise”
• Format of forecast uncertainty estimate can/must link three major areas– Numerical methods
• Must be able to produce chosen format– Forecasters
• Must be able to manipulate guidance– Must be simple and intuitive
– Users• Must find clear value in product given in chosen format
• Choice of exact format not critical– As long as software available to generate alternate formats
INTRODUCTION
PROPOSED NEW / LEGACY PRODUCTS TO BE MONITORED• Possible new probabilistic guidance products for high impact events
– Hydrometeorology• Extreme hydro-meteorological events, incl. dry and wet spells (CONUS)
– Exceedance of 1 & 5 inch thresholds– Exceedance of 6-hr flash flood guidance (or numerical outlook) thresholds
• Quantitative extreme river flow forecasting (OCONUS)– Tropical / winter storm prediction
• Extreme surface wind speed– Radii of gale, storm, hurricane force wind– Maximum surface wind (for intensity)
• Extreme precipitation (related to wet spells)• Storm surges
– Storm surge values along US Coast– Aviation forecasting
• Flight restriction• Icing, visibility, fog, clear air turbulence
– Health and public safety• Hot and cold spells
• “Legacy” internal probabilistic scores to assess long-term progress• With years of existing archive and/or • Can be easily recomputed
– General circulation• Probabilistic 1000 & 500 hPa height forecasts
– Tropical storm • Strike probability for track• Probability of intensity (central pressure or wind-based)
BACKGROUND
Issues to be discussed/worked out in this order:• NRC Recommendations
– Do we agree on major recommendations?• If not, let’s discuss any issues• If so, can we accept those as the basis/guidelines for our work
• 3rd Ensemble User Workshop Recommendations– Three major recommendations in response to NRC Report
• High level roadmap guiding our collaboration (Sept 2007)• New operational requirements (March 2008)• Design, develop, and gradually implement new forecast process
– Basic capabilities (2009-2010)– Full capabilities (2012-2017)
– Nine specific recommendations on design, development, implementation• Work areas identified, links to be worked out
– Continue development of expanded forecast process– Bias correction & downscaling methods for ensembles– Define summary ensemble information to be used (ST)– Contribute to establishment of NOAA-wide environmental data depository– Develop ensemble interrogation, modification, & product generation tools– Develop telecommunication facilities to access data– Develop unified NWS/NOAA probabilistic verification package (ST)– Develop & implement comprehensive training– Develop outreach program on use & communication of uncertainty
Do we need to modify/augment etc these recommendations?
NFUSE PRIORITIES?
49
COMPLETING THE FORECAST:ASSESSING AND COMMUNICATING FORECAST UNCERTAINTY
Zoltan Toth
Environmental Modeling CenterNOAA/NWS/NCEP
Acknowledgements:NRC Report; Louis Uccellini, Steve Lord & Ensemble Team
http://wwwt.emc.ncep.noaa.gov/gmb/ens/index.html
50
OUTLINE
• REVIEW OF 3RD ENSEMLBE USER WORKSHOP
• MAJOR RECOMMENDATIONS REGARDING NRC REPORT
• WHAT, WHY, AND HOW TO CHANGE?
• SPECIFIC RECOMMENDATIONS
• Logistics– Oct. 30 – Nov. 1 2006, Laurel, MD– Close to 100 participants
• NWS Regions (12), Headquarters (15), NCEP (37)• OAR (6), other government (4), private (4), academic (8) sectors & international (8)
– For further info, see: http://wwwt.emc.ncep.noaa.gov/gmb/ens/UserWkshop_Oct2006.html
• Topics– Assessing & propagating uncertainty throughout entire forecast process
• From observations to users• Working group discussions
– Ensemble configuration– Statistical post-processing– Data depository / Interrogation / Product generation/dissemination / Verification– User support / Outreach / Training
• Outcome– Enthusiastic discussions
• Convergence on number of topics• Open questions needing further research identified
– Strong support for sustained effort/engagement, annual meetings, etc• Recommendations
– Course of action in response to NRC report• Presented here first – will solicit and incorporate feedback from participants
3rd ENSEMBLE USER WORKSHOP
FOLLOW NRC REPORT RECOMMENDATIONSAll major recommendations embraced
• Develop roadmap for assessing & communicating forecast uncertainty– Based on science, technology, workforce considerations– Consistent with NOAA’s mission, NWS plans, embraced by “Enterprise”– Define end goal, 5-10 years horizon
• Adopt ensemble-based forecast process– Maximize forecast skill– Unify scientific, software, and technological infrastructure across NOAA
» Weather, water, climate applications
• Form high level planning and oversight team – March 2007– Each NWS office to delegate one representative– Team reports to Corporate Board
• Name programmatic and technical leads• Develop roadmap – Sept. 2007
MAJOR RECOMMENDATIONS - 1
• Develop roadmap for assessing & communicating forecast uncertainty
• Revise operational requirements to make them probabilistic – Make probabilistic format the primary requirement
• For each forecast application:– Replace single value/categorical format with probabilistic format as primary requirement– Revise/supplement corresponding performance measures (GPRA)
– Essential for orderly transition from traditional to new forecast process• NWS is requirement-driven organization
– Without clear new requirements, process is doomed– Phased implementation schedule in consultation with responsible offices
• Allows orderly transition to new requirements
• New requirements prepared by Planning & oversight team (or its designate)– Assisted by responsible NWS office– Presented to Corporate Board for approval – By March 2008
MAJOR RECOMMENDATIONS - 2
• Develop roadmap for assessing & communicating forecast uncertainty (.5 yr)• Revise operational requirements to make them probabilistic (1 year)
• Design, develop, and gradually implement new forecast process– Focus on missing scientific, technological, and human components
• Identify self-contained components with– Clear requirements and interfaces between components
– Define basic capabilities achievable in 2-3 years• Limited but consistent with and leading to end goal
– Full capabilities in 5-10 years
• Interface with NOAA THORPEX program– Research/development to improve skill & utility of probabilistic forecasts– Leverage related major NOAA, national, and international efforts
• Integrate with NWS, NOAA, national activities– New NOAA CONOPS process– NOAA-wide regional service plans– NUOPC planning/development
• Provide long-term funding support through PPBES– Overlap with W&W High Impact Event Theme Team
• Form development teams for specific tasks/components following workshp– Identify potential contributors within & outside of NWS & NOAA
MAJOR RECOMMENDATIONS - 3
• Major paradigm shift – Incorporate assessment and communication of uncertainty in forecast process
• Is it a major change in course of “Weather Ship”?– Ie, abandon course of ever improving single forecast scenario (expected value)?
• No – Expand, not abandon– Keep improving fidelity of forecasts, PLUS– Add new dimension
• Capture other possible scenarios – ensemble forecasting– Use a flotilla, instead of one ship, in exploring nature
– Existing activities are subset of expanded forecast process• Single value forecast is expected value of full probability distribution
– Can keep serving forecasts in old format to users who prefer that
PROPOSED CHANGE
Single forecast (driven by GFS winds) example for drifting virtual ice floe
Bob Grumbine, EMC
Initial position
7 September 2006
Ensemble forecast for drifting ice floe for same case
Bob Grumbine, EMC
Initial position
Most likely forecast for drifting ice floe for same case
Bob Grumbine, EMC
Initial position
• Why users (should) care about forecast uncertainty?– They admittedly want minimal or no uncertainty in forecasts
• Distinction between no uncertainty in the forecast, vs. not talking about it– Forecast uncertainty cannot be arbitrarily reduced
• Despite major ongoing & continuing efforts, they persist forever– Chaotic nature of atmosphere - land surface – ocean coupled system + initial/model errors– Level of uncertainty is determined by nature and level of sophistication in forecast system
– Forecast uncertainty can be ignored though• Negative consequence on informed users
– Not able to prepare for all possible outcomes» Assumes a certain scenario and remains vulnerable to others
• Possibly serious loss in social/economic value of forecast information
• Why forecasters (should) care about forecast uncertainty?– Imperfect forecasts are consistent w. observations (reliable) only if in prob format
• If in other format, must be brought into probabilistic format through – Verification / bias correction
WHY CHANGE IS NEEDED?
• More rationalized and enriched forecaster - user interactionsOld paradigm– Convoluted forecaster-user decision process
• User expects forecaster to make decision for them in presence of uncertainty– “Will it rain?” – “80%” – “But tell me, will it rain?”
New paradigm – Forecaster and user decision processes enhanced and better linked
• Allows forecasters to capture all knowledge about future conditions– Provision of information related to multiple decision levels in probabilistic format critical
» Provider helps interpret probabilistic info & and modify user decision process if needed» Option to continue providing single value or other limited info until user ready
• Allows users to decide about most beneficial course of action given all possibilities– Proper use of probability or other uncertainty information needed - Training
» User requests critical weather forecast info depending on their sensitivity
ADVANTAGES OF PROBABILISTIC FORMAT
• Focus on single forecast scenario – Reducing uncertainty in single forecast is main emphasis
• Loss of accuracy in forecast estimate of expected value of distribution– Mean of ensemble cloud provides better estimate
– Ignores or simplifies forecast uncertainty • Uncertainty assessed as statistically averaged error in single fcst (second thought)
– Ensemble cloud provides better estimate of case dependent variations in uncertainty– Use of single value / categorical forecast format
• Difficulty in formulating/communicating plausible alternate scenarios– Ensemble member forecasts can directly feed into Decision Support Systems
• One-way flow of information from observations to users– Not adaptable to case dependent user requirements
• Ensemble can propagate back user requirements to adaptive– Observing, assimilation, modeling/ensemble, post-processing and application components
» Applications in planning and execution of new CONOPS in high impact events
TRADITIONAL FORECAST PROCESS
z
Sin
gle
valu
e
Dis
tribu
tion
Ensemble Forecasting:Central role – bringing the pieces together
PROPAGATING FORECAST UNCERTAINTY
• Adopt ensemble approach across all environmental prediction activities– Expand forecasting with new dimension of uncertainty
• Multiple scenarios (in place of single scenario)– Provides best forecast estimate for both expected value (as before) and uncertainty (new)
– Unified scientific, technological, human approach• Sharing resources across NWS & NOAA
– Ensemble is centerpiece both symbolically and figuratively in forecast process• Ensembles act as a glue & two-way information channel
– Observing system, data assimilation, numerical modeling» ENSEMBLES
– Statistical post-processing, product generation, decision making
• Design, develop, & implement missing components of new forecast process– Gradual, measured steps
• Basic capability - Short-term, 2-3 yrs, leading to • Full implementation - Long-term, 5-10 yrs
HOW CAN IT BE DONE? NEW PARADIGM
SOCIOECON.
ENSEMBLES AND THE RESEARCH COMMUNITYLINKED THROUGH THORPEX – MAJOR INTERNATIONAL RESEARCH PROGRAM
GOAL: Accelerate improvements of high impact weather forecasts
SYSTEM
CLI
MA
TE F
OR
EC
AS
TIN
G /
CTB
ADAPTIVE COLLECTION & USE OF OBSERVATIONS
INTEGRATED DATA
ASSIMILATION & FORECASTING
USER CONTROLLABLE PROBABILISTIC FORECASTS
Days 15-60
NW
S O
PE
RA
TIO
NS
GLO
BA
L IN
TER
AC
TIV
E
FOR
EC
AS
T S
YS
TEM
(GIF
S)GLOBAL OPERATIONAL
TEST CEN
TER
GLO
BA
L OPER
ATIO
NA
L
TEST CENTER
WEATHER-CLIMATE LINK MODEL ERRORS
& HIGH IMPACT MODELING
1. Continue development of expanded forecast process– Focus on adaptive methods applicable for high impact events
• Collection & use of observations (targeted observations)• Data assimilation (case dependent background error estimation)• Numerical modeling (adaptive resolution & high impact modeling)• Ensemble forecasting (case dependent variations in membership & composition)• Decision support systems (flexible user actions depending on forecast probabilities)
2. Bias correction & downscaling methods for ensembles– Estimate/correct lead-time dependent bias in ensemble forecasts (on model
grid)– Generate fine resolution (NDFD grid) uncertainty/ensemble data
• Establish connection between (bias corrected) coarse model vs. fine user grids– Use reanalysis & hind-casts with operational systems as needed
3. Define summary ensemble information to be used to (ST)– Collapse vast amount of ensemble data for inclusion in expanded
NDFD/NDGD• E.g., 10, 50, and 90 percentile of forecast distribution (in place of single value)
– Manually inspect/modify summary ensemble information
SPECIFIC RECOMMENDATIONS – 1-3
ST – Short Term (2-3 yrs); LT – Long Term (5-10 yrs)
4. Contribute to establishment of NOAA-wide environmental data depository– Expand NDFD/NDGD database to include forecast uncertainty (ST)– Develop capability to hold all ensemble trajectories (LT)
• All members, variables, lead times
5. Develop ensemble interrogation, modification, & product generation tools to
– Derive summary information from ensemble (ST)– Manually modify summary ensemble info (ST)– Derive additional statistics from summary info (product generation, ST)– Automatically modify ensemble trajectories based on modified summary info
(LT)– Derive any info from full ensemble data (product generation, LT)
6. Develop telecommunication facilities to access data– Summary info & limited derived products (ST)– All ensemble forecasts & derived products (LT)
SPECIFIC RECOMMENDATIONS – 4-6
ST – Short Term (2-3 yrs); LT – Long Term (5-10 yrs)
7. Develop unified NWS/NOAA probabilistic verification package to (ST)– Assess statistical reliability and resolution for
• Computing official performance measures• Evaluating value added along forecast chain• Assessing value in newly developed vs. operational techniques
8. Develop & implement comprehensive training to– Prepare all participants for their new roles in expanded forecast process, incl.
• Statistical background• Ensemble methods• Best forecast practices in assessing uncertainty• Applications of probabilistic & other uncertainty information
9. Develop outreach program on use & communication of uncertainty– In partnership with entire Weather, Water & Climate Enterprise
• Determine best ways of communicating uncertainty• Compile sample of Decision Support Systems using uncertainty information
– Establish close partnership with public sector users (e.g., emergency, water management)• User feedback on new activities
– Explore how forecast process can be adapted according to user requirements
SPECIFIC RECOMMENDATIONS – 7-9
ST – Short Term (2-3 yrs); LT – Long Term (5-10 yrs)