USWRP Mesoscale Observing Networks Workshop8-10 December 2003

Summary of Discussions from the Advanced Modeling and Data Assimilation Working Group

Co-leaders: Steve Koch and Xiaolei ZouScribe: John Mecikalski

Identify the observing needs required to support research aimed at developing, testing, transitioning to operations improved forecast systems for severe weather (both cold and warm season), air quality, hydrology, chemical emergency response, and other applications.

Advanced Modeling and Data Assimilation Working Group Theme

Observational Needs for Numerical Modeling and Data Assimilation

1. Identify candidate surface-, airborne- and space-based observing systems appropriate to mesoscale measurement

2. Identify methods that could be used to design effective observing system3. Identify relevant modeling and information-delivery challenges4. Identify research and technological needs for development of adaptive and

targeted sampling strategies to support improvement of forecast systems

Definition of “Mesoscale” for the purposes of our discussion

• The group rejected use of a purely spatial definition of mesoscale (Orlanski 1976), but instead preferred the Lagrangian time scales suggested by Emanuel (1983) to place the lower (Brunt-Vaisalla) and upper (inertial period) limits on the problem - i.e., scales of 10 min - 12 h, and to put emphasis on observational requirements to properly sample mesoscale phenomena within that range of scales.

“Assimilation of meteorological or oceanographic observations can be described as the process through which all the available information is used in order to estimate as accurately as possible the state of the atmospheric or oceanic flow.”

Definition of Data Assimilation (Talagrand 1997)

Some Aspects of Data Assimilation

1. Weights assigned to observations are proportional to the accuracy of the measurements (defined as the inverse of the observational error covariances)

2. The accuracy and usefulness of an analysis in the variational framework is compromised when the observations and/or the background (model forecast) error covariances are poorly known, if there are biases, or if the observations and background errors are correlated.

3. The background error covariance matrix B in optimal interpolation is determined on the basis of separability of the horizontal and vertical correlations using gaussian functions with assumed scales - something that enhanced mesoscale observations might address. The “NMC method” is often used to estimate the B matrix in 3DVAR systems; while this offers a more global approach to the problem, it is questionably accurate at scales for which observations are lacking.

Top Four Questions Considered in Detail

1. What are the deficiencies in the current observational networks for various mesoscale user applications? Which additional and current observations will most effectively address these deficiencies?

2. What limitations do data assimilation methods impose on the effective use of observations?

3. What are the primary sources of model error? Which additional observations will most effectively reduce error?

4. What has been/can be learned from recent field experiments about the mix of observations needed to realize the greatest improvements in mesoscale data assimilation & forecasting, specifically:

a) Is it more cost effective to have intermittent, target observations at the mesoscale than enhancing the present operational networks to provide additional data in a continuous manner?

b) Is it more effective to sample the upper troposphere with fewer observing system than to sample the boundary layer with more observing systems for maximum inpact on mesoscale numerical prediction?

User Groups Identified(it would be beneficial to create a matrix of needs by user

group versus observation systems, and define testbed design)

Operational prediction centers

NWS/WFO/RFC (hydrology)

Surface transportation

Aviation

Homeland Security

Regional air quality

Energy

Water resources

Materiel testing

Weapons systems support

Fire weather

Agriculture

Recreation

What are the deficiencies in the current observational networks for various mesoscale user applications?

Which additional and current observations will most effectively address these deficiencies in the lower troposphere?

Precipitation rate (more accuracy & QC)

Hydrometeor types in 3D

Lower tropospheric 3D mass, winds, moisture fields @ 10 km hor. resolution

IFPS surface requirement < 2.5 km, hourly

Land surface properties: soil moisture and temperature profiles, snow cover and depth, SST, vegetation type/state updated daily

Turbulent flow and stability to 2 km altitude every 15 min

PBL turbulent fluxes (model validation), PBL heights, convective roll characteristics

Aerosols, chemical tracers, emissions data

Radiative transfer model requirements: detailed ozone, CO2, water vapor, clouds distribution

The most detailed observations are needed in geographically complex regions (such as coastal regions and mountainous regions)

Measurements at the surface may not represent well the PBL structure

What are the deficiencies in the current observational networks for various mesoscale user applications?

Which additional and current observations will most effectively address these deficiencies in the free troposphere and

stratosphere?Hydrometeor types (3D)

Tropopause topology ~10 km

Ozone profiling in the upper troposphere and stratosphere from balloons and aircraft

Mass, winds, moisture fields (3D): threshold = 100 km, objective = 10 km

Inadequate vertical resolution exists in satellite measurements (next slide)

Cloud heating rate profiles

Aerosols, chemical tracers, emissions data

Radiative transfer model requirements of detailed ozone, CO2, water vapor, clouds

Note: winds tend to be more effective in providing initial conditions for NWP models than mass data aloft, unless the phenomenon has a shallower vertical structure, in which case temperature data are more important

It is very difficult for models to retain surface observations with high spatial and temporal detail unless rapid updating (intermittent assimilation) is used

Upper level Measurement Needs: Satellites

• Vertical resolution needs cannot be met with either current or planned satellite sensor systems:– GPS Occultation has insufficient horizontal coverage

– GIFTS infrared interferometer lacks ability to sample cloudy atmospheres

– Oceanic measurements are inadequate for upper-level mesoscale phenomena that exert influence on the West Coast, and in fact, an effort should be made to assess this data void effect

– Microwave satellite use over land areas can complement IR techniques, but requires much better ground measurements of emissivity and TSFC

• Vertical temperature profiling is needed in cloudy regions using enhanced ground-based remote sensing systems - either currently in existence (AERI) or under development (slant-range GPS) or not yet proven (scanning multichannel radiometer)

• Better wind measurements are also needed. MODIS at high latitudes might help. Measurements from regional jet airliners could also help. Improved vertical resolution is likely from GIFTS for wind determination.

Upper level Measurement Needs: Sampling• Having measurements at ~100-km resolution could help with

background error covariance estimation and analysis of such upper-level mesoscale phenomena as jet streaks, the largest gravity waves, and cold fronts aloft, but would mean ~1000 profiling sites, and would be inadequate for other mesoscale phenomena of smaller scales

• Temporal sampling requirements are not as demanding as for the PBL (1 - 3 hour increments may suffice)

• Elevated convection remains extremely challenging. Deep convection observing requirements are primarily in the 0 - 12 h frame (especially in the 0 - 6 h nowcasting frame). Since deep convection usually has limited areal extent, targeted observations may be of some help.

• For meso-alpha scale systems (e.g., heavy snowbands), data collection upstream of the event 12 - 24 h ahead may be most useful for improving forecasts. Future hyperspectral satellite measurements and a100-km resolution national tropospheric profiling network may be very useful in this regard.

GEMS

• GEMS (Global Environmental Micro Sensors) is a10-20 year in situ sensor system development plan being proposed by ENSCO, Inc. and partners to fill the large data gap globally throughout the entire troposphere and lower stratosphere

• GEMS promises to be able to produce measurements of the state variables with global coverage at 1-km resolution, according to ongoing feasibility studies (this would require 109 probes)

• Targeted systems (UAVs, constant altitude balloon swarms, aircraft) could be used to dispense GEMS sensors where needed

• GEMS technology is not presently available. There are many unresolved issues related to communications, navigation, power requirements, etc. However, investment in this longer-term research idea may well prove rewarding.

• Is it more effective to deploy intermittent, targeted observations at the mesoscale or to enhance the present operational networks to provide additional data in a continuous manner with coarser spatial resolution?

• Do enhanced observations in the upper troposphere have a greater or lesser positive impact on mesoscale analysis and prediction than even greater sampling of the boundary layer with more observing systems (the PBL being easier and less costly to observe than the upper troposphere)?

What has been (or can be) learned from recent field experiments about the mix of observations needed to realize the greatest improvements in mesoscale data

assimilation & forecasting?

1. Height and temperature fields can be derived from a network of observations of the winds (e.g., profilers) at scales for which the hydrostatic assumption is valid. However, vertical motion estimation is quite problematic.

2. As the scales become shorter, the need for independent observations of both the mass and wind fields becomes increasingly greater as no balance criterion exists. Storm-scale retrievals are possible under special conditions (volume scan intervals < ~3 minutes and a sufficient volume of scatterers). Rapid scanning may be possible with proposed phased-array radar and/or CASA-type radars.

3. Static (intermittent) data assimilation schemes may not work as well for mesoscale data assimilation as dynamic 4DDA.

4. An important lesson learned from past field experiments is the need to have four-dimensional distribution of accurate in situ observations in order for the data assimilation scheme to retain the information.

5. Point measurements are useful for data assimilation, provided that there are a sufficient number of such observations, but they are generally more useful for validation of other observations (e.g., satellite products). Also, assimilation of single point measurements assumes the existence of an appropriate structure (or objective analysis influence radius) function

Field Experiments and Data Assimilation

Inviscid form of the divergence equation is used to retrieve geopotential heights from profiler winds on Lambert conformal grid (m = map factor)

Virtual temperature is calculated hypsometrically from retrieved heights

m22 fv

x

u

y

2m2 v

x

u

y

u

x

v

y

mu

mx

u

p

y

v

p

m2D2 D

t m2 u

D

x v

D

y

D

p

R Tv d ln pPu

Pl zu zl

Observational needs: Targeted Observations• Targeted observations are probably not useful for mesoscale data

assimilation because they must be repeatedly taken over a domain of appreciable size and at multiple levels to avoid having the mesoscale model reject the information.

• This problem is compounded by the fact that targeted observations they can take longer to collect than is useful for short-term forecasts. Examples include the use of UAVs, dropsondes, etc.

• Also, the use of singular vectors and other techniques to predict the regions of maximum sensitivity to analysis errors (deficiencies) is not a proven technique at the mesoscale.

• Instead, we suggest focus be placed on radar and satellite targeting :– Extend the GOES Rapid-Scan strategy to include targeted observing based

on forecasts of where greatest sensitivity lies– Alter the scanning strategies for the WSR-88D radar to include greater

resolution near the surface, more variable elevation choice options, and perhaps sector scans

What are the primary sources of model error? Which additional observations will most effectively reduce error?

• The diurnal cycle is poorly forecast. Improvements may require better definition of land surface fields, radiation, PBL heights.

• PBL parameterization schemes have difficulty predicting vertical mixing correctly, and they suffer from nonlinear interactions with other model physics masking potential problems. This suggests the need for observed profiles of turbulent fluxes and PBL height.

• Cloud distribution (depth, fraction, etc.) and interactions with radiation remains challenging. Detailed analysis of hydrometeor fields, consistent with model’s cloud microphysics, is needed.

• Analysis of vertical motion and/or thermodynamic fields consistent with grid-resolved cloud fields in the initial state is needed for short-range precipitation improvements (diabatic initialization). This may be quite a different problem for parameterized convection.

• Lack of mesoscale model validation data on a continuing basis

What limitations do data assimilation methods impose on the effective use of observations?

Negligible spatial correlation in observational errors is typically assumed in 3DVAR technique. A spatial averaging “solution” to this problem demands better quality, independent, higher resolution data.

Variational data assimilation schemes must make effective use of surface observations. Observation nudging is an effective method for mesoscale data assimilation. However, this method cannot easily use indirect observations (e.g., satellite radiances). A combination of nudging and variational approaches is recommended for greater generality.

The nowcasting requirement to be able to maintain strong horizontal gradients in the short-range forecast is a challenge for 3DVAR.

There is a great need for mesoscale observational data that can be used to determine background error covariance structures in variational data analysis systems (e.g., IHOP kinds of data).

Testbeds can be established to design the “optimal” data assimilation system and address known model deficiencies for specific end-user needs

Ensemble Kalman filter (EnKF)/smoother Approach for Mesoscale Data Assimilation

Advantages: The EnKF approach employs an ensemble of data assimilation

cycles carried out simultaneously using different sets of randomly perturbed observations to create each of ~25 members

Flow-dependent background error covariance can be obtained Does not require the development of a linear and adjoint model Does not require the linearization of the evolution of the forecast

error covariance (nonlinearity is maintained)

Limitations: Do ensemble members actually describe real background

covariance structures? Spin-up problem (for prediction of short-live mesoscale system) Need demonstration with real data at the mesoscale (so far, tests

have been restricted to idealized cloud simulations)

Four-dimensional Variational Approach for Mesoscale Data Assimilation

Advantages: The control variable (the variable with respect to which the cost function is

minimized) is the initial state of the NWP model; thus, the model is used as a strong constraint

Summation of the cost function over time for each observational increment is computed with respect to the model integrated to the time of the observation. Thus, 4DVAR seeks an initial condition such that the forecast best fits the observations within the assimilation interval.

The 4DVAR analysis at the final time is identical to that of the extremely expensive extended Kalman filter method, provided that we can assume that the model is perfect and that we know the error covariance at the initial time (i.e., 4DVAR can evolve the forecast error covariance ahead).

Limitations: Since 4DVAR assumes a perfect model, it will give the same weight to

older observations at the beginning of the assimilation interval as to newer observations at the end of the interval.

This technique is computationally expensive and requires the computation of the gradient at every iteration and integration of these weighted increments back to the initial time using an adjoint model.

Observational needs• Construct and examine the role of mesoscale flow-dependent and time-varying

background error covariance

• Do not derive “observations” with strong background input to avoid correlation between observations and background

• Quantify observational error statistics:- Covariance structures in both the horizontal and vertical

- Bias removal - Instrumental errors (including signals in the true atmosphere not resolved by

the instrument)- Representative errors (grid scale dependent)- Other errors (balloon drift, …)

• Quantify error statistics of signals in the observations that are not resolved by data assimilation at a specified resolution

• Determine forward model errors (e.g., fast radiative transfer model, terrain mismatch, etc.)