Physics Test Harness

Using the Global Modeling TestBed Single Column Model to Test a Newly Developed Convective Parameterization

Motivation The initial focus of the Global Modeling TestBed (GMTB), a collaborative project between NOAA GSD and NCAR through the Developmental Testbed Center, is to develop a framework to evaluate advancements in physics parameterizations for future use in operational NWP. Such a framework consists of an Interoperable Physics Driver (IPD), a Common Community Physics Package (CCPP), and a physics test harness. The physics driver provides a common interface for physics packages, the test harness provides a uniform testing and evaluation functionality, and the CCPP provides a repository of supported physics suites to the research and operational communities. The purpose of this poster is to report on the initial test of one component of the physics test harness, the GMTB single column model.

Grant J. Firl

* grantf@ucar.edu

• Common infrastructure for testing physics development

• Simple-to-complex progression, conceptually and computationally

• Researchers can “enter” test harness at whichever level is appropriate

Tools and Data Provided by DTC for Physics Test Harness • documentation and access to IPD and CCPP code• test case catalog with initialization and forcing data, observational data

for comparison, benchmark data from operational physics suites• support for using SCM and global model workflow• basic plotting and evaluation routines

New Scale-aware

Convection

New PDF-based Turbulence and

Convection

Developer uses provided documentation to connect scheme with IPD and modify a physics suite in the CCPP with the new scheme

PhysicsTest

Harness

PhysicsReview

Committee

CCPP

Basic Steps1.Run scheme with

modified CCPP suite through test harness

2.Compare modified suite to operational benchmarks and observations

3.Track computational performance compared to control physics suite

The Physics Review Committee curates the schemes within the CCPP to include a small subset of high-performing schemes for multiple NWP applications

Outcomes • Thorough testing of new scheme leads to targeted continued development

•Scheme performs better than control and warrants further investigation by operational centers

References Davies, L., Jakob, C., Cheung, K., Genio, A. D., Hill, A., Hume, T., Keane, R. J., Komori, T., Larson, V. E., Lin, Y., Liu, X., Nielsen, B. J., Petch, J., Plant, R. S., Singh, M. S., Shi, X., Song,

X., Wang, W., Whitall, M. A., Wolf, A., Xie, S., and Zhang, G. A single-column model ensemble approach applied to the TWP-ICE experiment. J. of Geophys. Res.-Atmos 118, 12 (2013), 6544–6563.

Fridlind, A. M., Ackerman, A. S., Chaboureau, J.-P., Fan, J., Grabowski, W. W., Hill, A. A., Jones, T. R., Khaiyer, M. M., Liu, G., Minnis, P., Morrison, H., Nguyen, L., Park, S., Petch, J. C., Pinty, J.- P., Schumacher, C., Shipway, B. J., Varble, A. C., Wu, X., Xie, S., and Zhang, M. A comparison of TWP-ICE observational data with cloud-resolving model results. J. Geophys. Res.-Atmos. 117, D5 (2012).

Grell, G.A, and S. R. Freitas: A scale and aerosol aware stochastic convective parameterization for weather and air quality modeling. Atmos. Chem. Phys., 14, (2014), 5233- 5250.Taylor, K. Summarizing multiple aspects of model performance in a single diagram. J. Geophys. Res.-Atmos. 106, D7 (2001), 7183–7192.

SCM Description

Research-to-Operations Pipeline

New Aerosol-coupled Double Moment

Microphysics

grantf@ucar.edu

Summary • SCM that works with an interoperable physics driver as part of a physics test harness

has been developed and can be a valuable tool for physics developers• G-F scheme seems to reduce the dry bias from operational GFS scheme in PBL• G-F scheme exhibits generally higher skill scores despite it’s “untuned” state• G-F scheme generates a more varied response to the forcing ensemble• G-F produces weaker convective tendencies, leaving grid-scale MP to do more work• G-F mass fluxes have different shape (and weaker) than SAS• G-F has lower convective precip. ratio (and interesting response to forcing strength)• The sensitivity to the closure choice in the G-F scheme is greater than the

improvement indicated over the operational GFS configuration

Results

Results • Uses IPD to connect to operational

GFS physics suite• Portable code uses minimal

dependencies • Updated to reflect ongoing changes in

IPD and GFS physics• Driven from observationally-based

cases (GCSS-style)• Features complete documentation and

User’s GuideAvailable to friendly

users on NOAA’s VLab

Case Setup • GCSS case built from ARM TWP-ICE field campaign data• Features deep and suppressed convection near Darwin Australia during 1/20/2006 - 2/12/2006

• 64 hybrid-sigma levels, 10 min △t• Forced by fixed SST, prescribed hor.

advective tendencies of T, q, prescribed vertical velocity, and nudged u, v

• Uses 100-member forcing ensemble that varies forcing based on uncertainty in precipitation measurement

• Analysis follows Fridland et al. (2012) and Davies et al (2013)

http://www.dtcenter.org/GMTB/gmtb_scm_doc/

Physics Scheme Control / Grell-Freitas

Surface NOAH (ocean surface)

Radiation RRTMG

PBL Hybrid EDMF

Microphysics Zhao-Carr

Deep & Shallow Convection SAS (latest)/ Grell-Freitas

�0.005 0.000 0.005 0.010 0.015 0.020 0.025

q (kg/kg)

0

20000

40000

60000

80000

p(P

a)

SAS (0.963)GF-0 (0.988)

0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

cloud fraction

0

20000

40000

60000

80000p

(Pa)

SAS (0.086)GF-0 (0.104)

S =4 (1 +R)4

⇣�f + 1

�f

⌘2(1 +R0)

4

�4 �3 �2 �1 0 1 2 3 4 5

g/kg/day

0

20000

40000

60000

80000

p(P

a)

SASGF-0

forcePBLDeep ConShal ConMP

�20 �15 �10 �5 0 5 10 15

K/day

0

20000

40000

60000

80000

p(P

a)

SASGF-0

forcePBLDeep ConShal ConMPLWSW

�12 �10 �8 �6 �4 �2 0 2

conv. q tendency (g/kg/day)

0

20000

40000

60000

80000

p(P

a)

SASGF-0

�5 �4 �3 �2 �1 0 1 2 3 4

MP q tendency (g/kg/day)

0

20000

40000

60000

80000

p(P

a)

SASGF-0

Jan 20 2006 Jan 21 2006 Jan 22 2006 Jan 23 2006 Jan 24 2006 Jan 25 2006date

0

2

4

6

8

10

12

prec

ip(m

m/h

r)

SAS (0.727)GF-0 (0.782)

Jan 20 2006 Jan 21 2006 Jan 22 2006 Jan 23 2006 Jan 24 2006 Jan 25 2006date

0.5

0.6

0.7

0.8

0.9

1.0

1.1

RH

at50

0H

Pa

SAS (0.813)GF-0 (0.773)

0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 0.45

kg/m2/s

0

20000

40000

60000

80000

p(P

a)

SASGF-0

updraftdowndraftdetrainment

0.0 0.5 1.0 1.5 2.0 2.5 3.0

total surface precipitation (mm/hr)

0.0

0.2

0.4

0.6

0.8

1.0

conv

ectiv

epr

ecip

itatio

nra

tio

0.00 0.05 0.10 0.15 0.20

kg/m2/s

0

20000

40000

60000

80000

p(P

a)

SASGF-ctlGF-0GF-2GF-5GF-13

updraftdowndraftdetrainment

�0.005 0.000 0.005 0.010 0.015 0.020 0.025

q (kg/kg)

0

20000

40000

60000

80000

p(P

a)

SAS (0.962)GF-ctl (0.988)GF-0 (0.985)GF-2 (0.998)GF-5 (0.996)GF-13 (0.990)

Jan 20 2006 Jan 21 2006 Jan 22 2006 Jan 23 2006 Jan 24 2006 Jan 25 2006date

0

1

2

3

4

5

6

7

8

prec

ip(m

m/h

r)

SAS (0.720)GF-ctl (0.766)GF-0 (0.762)GF-2 (0.836)GF-5 (0.836)GF-13 (0.857)

Acknowledgements This work is a collaboration of the Developmental Testbed Center; the GMTB project is funded by NOAA’s Next-Generation Global Prediction System (NGGPS).

Active Convective PeriodMean Profiles

Red = GFS operational suite (with SAS conv.)Blue = Modified suite with Grell-Freitas conv.

Taylor (2001) skill scores in legends:

Shading denotes 10-90% of forcing ensemble members

G-F improves dry bias in PBL

Specific HumidityCloud Fraction

G-F improves low cloud fraction

Total Sfc. Precip. Rate RH @ 500 hPa

Time SeriesG-F slightly improves skill score for total sfc. precip.

Obs. of mid-tropospheric humidity fall within ensemble range suggesting that SCM errors are due to physics instead of erroneous forcing

Main Differences

Sensitivity to Grell-Freitas Closure Choice

Specific Humidity Tendencies Temperature TendenciesG-F produces weaker convective tendencies, leaving the grid-scale microphysics scheme to do more “work” to balance the forcing. There are also minor changes to the PBL scheme tendencies.

Conv. Moisture Tendency Micro. Moisture Tendency Conv. Mass FluxesAlthough weaker, the G-F scheme’s response to the forcing ensemble is more varied

The shape of the mass flux profiles produced by the conv. schemes is quite different

DAVIES ET AL.: ENSEMBLE SCM STUDY FOR TWP-ICE

−10 −8 −6 −4 −2 0 2 4 6 8 10

200

300

400

500

600

700

800

900

1000

Pre

ssur

e (h

Pa)

Mean best−estimate−ensemble mean RH

Figure 6. Mean period-averaged difference between thebest estimate and ensemble mean relative humidity for theactive period for each SCM.

active period and 0.5 to 1 in the suppressed period. In theactive period, the models also show a very diverse behaviorwith forcing strength, with some showing an increase in CPF(e.g., GISS, UM-GR), some showing a near-constant CPF(e.g., NCEP, SCAM), and some showing a decrease (e.g.,UM-PC). The GFDL2 model shows a somewhat erraticbehavior. Models of the same type show different behav-ior depending on the parameterization scheme used (e.g.,UM-PC versus UM-GR).

[45] In the suppressed period, all SCMs have a CPF ofgreater than 50%. There is a tendency in almost all mod-els for the CPF to increase with increasing forcing althoughthere is much scatter in the relationship. There are twogroups of models, with either very high or relatively lowCPF. There is some consistency between the periods, withthe GISS and UM-PC models showing the lowest CPFin both.

[46] The rather wide spread in model behavior is likelyindicative of large differences in the assumptions made in thedifferent convection treatments on how to partition rainfall

between convection and the larger scales. As this will likelyhave an impact on the vertical distribution of heating andmoistening, an important issue for future work is to provideobservational constraints for the relationships shown here.3.1.3. Ensemble Moisture Budget Characteristics

[47] The ensemble provides an opportunity to investigatethe interplay between modeled moisture and the moisturebudget terms. In particular, this study permits a compari-son between how the models control their moisture budgets.Given that the models are forced by prescribing horizon-tal advection terms and vertical velocity, they independentlydevelop moisture budget terms such as vertical advectionterms and moisture convergence in addition to the moisturecontributions from parametrized processes such as convec-tion and surface evaporation. This is an important differencebetween this study and previous intercomparisons [e.g.,Woolnough et al., 2010; Guichard et al., 2004] where thetotal moisture forcing was prescribed. Furthermore, giventhat this study also includes both best estimate and ensemblesimulations, comparison can be made about the additionalmodel characteristics exposed using an ensemble comparedto a single best estimate simulation.

[48] Figure 8 shows time average precipitable wateragainst various terms in the moisture budget for the activeperiod for all models and ensembles in this study. Verysimilar results are obtained for the suppressed period (notshown). Figure 8a shows that during the active period, theSCMs tend to divide into models in which lower precip-itable water is associated with larger precipitation (GISS andSCAM), models where precipitable water is higher for largervalues of precipitation (UM and CLUBB), and those models,including CRM, where precipitation is independent of pre-cipitable water. The GFDL model is somewhat an exceptionas its relationship shows significant scatter.

[49] The largest term in the moisture budget is the mois-ture convergence term which is shown in Figure 8b. Inall models, the moisture convergence term shows a similarmagnitude and characteristics to precipitation which is notsurprising as it is the largest source of moisture for the gridbox exceeding surface evaporation by an order of magnitude(see below). Furthermore, Figure 8b shows that the moisture

Figure 7. Time-averaged scatter plots of surface precipitation against convective precipitation (shownas a fraction of the total surface precipitation) over the active (left) and suppressed (right) periods forensemble members 10–90. Each model type is represented by a color and each model of a given type bya symbol. The multimodel best estimate ensemble is represented by a large asterisk. The CLUBB modeland CRM do not submit partitioned precipitation data.

6553

Total Sfc. Precip. vs. Conv. Precip. Ratio(each point represents the mean value for one ensemble member)

Same plot from SCM intercomparison study: Davies et al. (2013)

The ratio of convective precipitation to total precipitation is much smaller in G-F modified suite, especially for the more weakly forced runs

light blue points represent GFS physics as of 2011

Specific Humidity Total Sfc. Precip. Rate Conv. Mass Fluxes

A SCM can be a valuable tool for determining sensitivities to parameterization options. For the Grell-Frietas sheme, this case was most sensitive to the parameter named “ichoice” which controls the type of closure used in the scheme.

mass flux profiles are very sensitive to this choice

ichoice=2 or 13 produce the highest skill scores