Upload
aloha
View
62
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Evaluation and Comparison of Multiple Convection-Allowing Ensembles Examined in Recent HWT Spring Forecasting Experiments. Israel Jirak, Steve Weiss, and Chris Melick Storm Prediction Center. Convection-Allowing Ensembles Overview. - PowerPoint PPT Presentation
Citation preview
WoF Workshop, April 3, 2014
Evaluation and Comparison of Multiple Convection-Allowing Ensembles Examined in Recent HWT Spring Forecasting Experiments
Israel Jirak, Steve Weiss, and Chris Melick Storm Prediction Center
• Convection-allowing ensembles (~4-km grid spacing) can provide important information to forecasters regarding the uncertainty of storm intensity, mode, location, timing, etc. on the outlook to watch scale
• These ensembles will play an important role in the ability of SPC to provide a more continuous flow of probabilistic hazard information in support of WoF2 March 2012 29 June 2012
24-h neighborhood prob UH ≥25 m2/s2 24-h ensemble max 10-m Wind Speed
Convection-Allowing EnsemblesOverview
• An experimental real-time Storm-Scale Ensemble Forecast (SSEF) system has been produced for the NOAA Hazardous Weather Testbed (HWT) Spring Forecasting Experiment (SFE) by OU CAPS since 2007 through CSTAR funding– Comprised of 4-km convection-allowing WRF and ARPS members:
• 2007: 10 members; 2008: 10 members; 2009: 20 members; 2010: 26 members (full CONUS to 30 h); 2011: 50 members (full CONUS to 36 h); 2012: 28 members; 2013: 27 members at 00Z (to 48 hours) and 8 members at 12Z (to 18 hours)
– Primarily examine explicit storm attributes, especially hourly maximum fields (HMFs): Updraft Helicity, Updraft Speed, 10-m AGL Wind Speed, 1-km AGL Reflectivity
– Ensemble display approaches include spaghetti plots, ensemble max, and neighborhood probabilities (more on next slide)
• SPC began processing deterministic high-resolution runs from EMC and NSSL as the Storm-Scale Ensemble of Opportunity (SSEO) in 2011
• The 4-km AFWA ensemble was made available to SPC in 2012
Convection-Allowing EnsemblesHistory
• Traditional ensemble probabilities of HMFs from high-resolution models are not especially useful, owing to poor agreement among members at the grid point of these fields.
• Applying a binary neighborhood approach to a storm-scale ensemble improves the statistical results of HMFs in forecasting severe weather– ROI=20-40 km– Sigma=30 grid points
• Same approach can also be applied to observations (e.g., radar reflectivity) for verification purposes
03 May 2008 (Harless 2010)
Convection-Allowing EnsemblesNeighborhood Probabilities
Convection-Allowing EnsemblesNeighborhood Probabilities
HM Updraft Helicity > 25 m2s-2
SSEO 24-hr fcst valid 00Z 28 April
Convection-Allowing EnsemblesNeighborhood Probabilities
Grid-Point Probability HM Updraft Helicity >25 m2s-2 SSEO 24-hr fcst valid 00Z 28 April
Convection-Allowing EnsemblesNeighborhood Probabilities
40-km Neighborhood Probability HM Updraft Helicity >25 m2s-2 SSEO 24-hr fcst valid 00Z 28 April
Convection-Allowing EnsemblesNeighborhood Probabilities
40-km Neighborhood Smoothed Prob HM Updraft Helicity >25 m2s-2 SSEO 24-hr fcst valid 00Z 28 April
• OU/CAPS Storm-Scale Ensemble Forecast (SSEF) System• Since 2007; 36-hr forecasts from 00z; 12z runs began in 2013• Primarily WRF-ARW; 4-km grid spacing; forecasts to 60hrs in 2014 • Multi-physics, multi-initial conditions: applies SREF perturbations to NAM ICs• Advanced physics, 3DVAR & radar data assimilation; available for HWT/SFE
• SPC Storm-Scale Ensemble of Opportunity (SSEO)• Since 2011; 36-hr forecasts at 00z & 12z; 7 members (2 time-lagged) • Multi-model (ARW, WRF-NMM & NMM-B), multi-physics; ~4-km grid spacing• Uses available deterministic models at SPC to process as an ensemble• Basic data assimilation through NDAS; available year-round in SPC
• Air Force Weather Agency (AFWA) Ensemble• Since 2012; 60-hr forecasts at 00z & 12z; 10 members; 4-km grid spacing• Single model (WRF-ARW), multi-physics, multi-initial conditions• Cold start from downscaled global model forecasts (GFS, UM, CMC)• No data assimilation; available year-round in SPC
Convection-Allowing EnsemblesSystem Comparison
• The Fractions Skill Score (FSS) was calculated for neighborhood probability (ROI=40 km; σ=40 km) of updraft helicity ≥ 25 m2s-2 for the SSEO/SSEF versus practically perfect hindcasts of preliminary severe weather reports (ROI=40 km; σ=120 km) during SE2011
• The SSEO had higher fractions skill score (FSS) for neighborhood probabilities of UH ≥25 m2/s2 during SFE2011 than the SSEF
• The number of members included in the SSEF did not seem to have a strong impact on the statistical results for neighborhood probabilities of UH during SE2011 when verified against severe weather reports
Convection-Allowing EnsemblesSFE2011 Results
Nprob UH ≥25 m2/s2SSEO SSEF – 24 memFSS = 0.84 FSS = 0.68
3-hr [NPRS]:UH ≥25 m2s-2 valid 06Z on 02 June 2011w/ verifying reports and practically perfect hindcast
FSS 3-h periods SFE2011
• Even for 6-h QPF, the SSEO received the highest subjective ratings relative to other operational and experimental models and ensembles during SFE2011
• Statistically, the probabilistic QPF forecasts (>0.5”) from the SSEO were typically as good as (if not better than) the SSEF during SFE2011 at various lead times
Convection-Allowing EnsemblesSFE2011 Results
from Tara Jensen, DTCfrom Dave Novak, WPC
SSEO favored over CAPS ensemble
• During SFE2012, the SSEO outperformed the SSEF and AFWA in terms of FSS for neighborhood probabilities of reflectivity ≥40 dBZ (bug later found in SSEF)
• Subjective ratings by the SFE2012 participants of HMF ensemble forecasts tended to favor the SSEO forecasts of UH over the SSEF and AFWA forecasts
Convection-Allowing EnsemblesSFE2012 Results
Hourly FSS Nprob Refl ≥40 dBZ 3-hr ensemble forecast ratings
(max, nprob) of UH
• The quality of the AFWA forecasts was less consistent than the SSEO forecasts
• Some UH forecasts from the AFWA ensemble were very good (bottom left) while others were poor (bottom right)
Convection-Allowing EnsemblesSFE2012 Results
• The impact of radar data assimilation in the CAPS SSEF was evident in the first 4 hours of the 12 UTC-initialized forecast.
• Otherwise, there was little statistical difference in the FSS among the 00 and 12 UTC SSEO and SSEF.
• Subjective ratings of 00Z ensemble HMFs were again favorable for the SSEO during SFE2013
Convection-Allowing EnsemblesSFE2013 Results
Hourly FSS Nprob Refl ≥40 dBZ
3-hr ensemble HMF forecast ratings
(max, nprob)
• Why is a “poor man’s ensemble” (i.e., SSEO) performing as well as formally designed ensembles? Let’s consider some aspects of configuration for convection-allowing ensembles– Single model vs. multi-model– Number of members– Initial conditions and IC/LBC perturbations– Physics
Convection-Allowing EnsemblesConfiguration
• Even with the same initial conditions, clustering often occurs by model core
• Generally more confident in a solution if different model cores are in agreement
• Is a multi-model approach a good way to address uncertainty in a convection-allowing ensemble?
Convection-Allowing EnsemblesConfiguration: Single model vs. multi-model
WRF-NMM WRF-ARW
SSEO
21Z on 16 April 2011
3-hr spaghetti plot of UH ≥25 m2s2
• Is the success of the SSEO a fortuitous balance of members with an underforecast bias and those with an overforecast bias (Row and Correia, 2014 AMS); not necessarily a result of using multiple model cores?
• Neighborhood verification of radar reflectivity reveals members with lower biases (e.g., NAM Nest) versus those with higher biases (e.g., NSSL-WRF)
• Biases will change with upcoming HiResW upgrade, so we may learn more this spring
Convection-Allowing EnsemblesConfiguration: Single model vs. multi-model
• For the way convection-allowing ensembles are currently configured, there does not appear to be a huge benefit to running more than ~10 members– Clark et al. (2011) objectively identified the “point of diminishing returns” at 9
members for 6-hr QPF at f30 and 2-km scale
Convection-Allowing EnsemblesConfiguration: Number of members
• Could run additional members to more effectively sample the forecast PDF, but is it worth the additional computational cost? Use a larger neighborhood?
from Clark et al. (2011)
• Currently, all members of the SSEO are initialized from the NAM (including two time-lagged members), so diversity primarily arises from multi-model/physics
• AFWA approach (single model, 3 different IC/LBCs) often leads to higher, overconfident probabilities
• SSEF approach not an obvious improvement over single, unperturbed IC (i.e. SSEO), suggesting ICs not properly perturbed at this scale
Convection-Allowing EnsemblesConfiguration: Initial conditions and IC/LBC perturbations
AFWA OBS
high probsnothing observed
• In four test runs for May 2013, Kong et al. (2014) found larger domain-average ensemble spread for multiple fields by directly using LBCs from SREF members rather than extracting perturbations and applying to the NAM (current strategy)
• NSSL-WRF ensemble this spring will directly utilize IC/LBCs from selected SREF members
Convection-Allowing EnsemblesConfiguration: Initial conditions and IC/LBC perturbations
Ense
mbl
e Sp
read
from Kong et al. (2014)
• Though spread in an ensemble with physics-only diversity is less than that from an ensemble that also includes IC/LBC perturbations, the contribution to spread from physics diversity can be large, including for instability fields (Clark et al. 2010)
Convection-Allowing EnsemblesConfiguration: Physics diversity
from Clark et al. (2010)
• Convection-allowing ensembles (~4-km grid spacing) can provide useful information to forecasters regarding the uncertainty of storm intensity, mode, location, timing, etc. in generating outlooks on Day 1; setting the stage for the continuous flow of probabilistic hazard information down toward the warning scale
• The SPC SSEO has proven to be as useful/skillful as formally designed convection-allowing ensembles, which raises questions about effective/proper configuration of these types of systems
• NSSL is running eight 00Z members this spring with only IC/LBC diversity directly from 21Z SREF members
• CAPS is planning to run an experimental 4-km EnKF SSEF system this year in near real-time for comparison with traditional SSEF forecasts
Convection-Allowing EnsemblesSummary