Academic Developer’s Perspective

Talât OdmanGeorgia Institute of Technology

and Amir Hakami

Carleton University, Canada

9th Annual CMAS Conference

October 14th, 2010

State of Academic Development

• There is not much funding out there for model development– Model development tasks are not viewed favorably in

research proposals– General opinion is that CMAQ ≈ EPAMAQ in terms of

ownership and development responsibility• Mechanisms for rapid transition from research to

operations are lacking• Despite limited resources, model development

continues at universities, under different names – We have to maximize the use of those limited resources

• Universities need help from the Community– To promote the role of universities in model development– To make model development easier for university

researchers

Convoluted Model Development

• There are two types of model development1. Modular (e.g., ISORROPIA, APT)2. Convoluted (e.g., DDM, Adaptive Grid, Adjoint)

• In CMAQ, there is an assumption by design that all development will be modular

• Convoluted development takes time– In the past, a new version of CMAQ was being released

every year – Some convoluted developments became obsolete before

completion; others are still subject to the same risk• DDM is a success story but is it an exception?

– It was developed to a large extent in another model– It enjoyed exceptional Community (C = C) support– Developers dispersed into the Community; many of them

continue the development effort

Stories of Adaptive Grid and Adjoint

Adaptive Grid Adjoint

Driving application(s)

Land management and forecasting

Forecasting, health, and decision support

Type of academic development

2 PIs Fairly scattered to multiple universities

EPA Support Funding (ceased in 2003)

Development

Other CommunitySupport

DoD and EPRI funding

API funding

Current CMAQ version

4.5 4.7

How can we make convoluted development easier?

• Support of current variables should continue in new versions– Primitive variables should be preferred since they are

less likely to be removed (e.g., density instead of a lumped quantity that contains density)

• No functionality should be removed without notice– Emission file formats changed in SMOKE– CMAQ version 4.7 removed the RADM cloud– Good to see “backward compatibility” as a priority

• New additions should be well documented– Peer reviewed publications– Updates to the science document

CMAQ’s coordinate system is difficult to comprehend

• In CMAQ’s horizontal diffusion, the Smagorinsky parameterization assumes Cartesian coordinates.

• Becker and Burkhardt (2007, Monthly Weather Review, 135, 1439-1454) revisited Smogarinsky’s mixing-length based parameterization for spherical and terrain following vertical coordinates of a GCM.

• The Smagorinsky parameterization must be derived for CMAQ’s generalized coordinates. Meanwhile, by intuition, I proposed

Watch out hemispheric modelers!

• There may be a directional bias in horizontal diffusion when the map scales display a wide range over the modeling domain such as in hemispherical applications with polar stereographic coordinates.

0 10 20 30 40 50 60 70 80 900.8

0.9

1

1.1

1.2

1.3

1.4

1.5

1.6

1.7

1.8m vs latitude

latitude [degrees]

m

What should we do about the generalized coordinate system?

• At a minimum , we should add divergence and vorticity to the “model provided variables” list– This way, these coordinate dependent variables can be

used directly in parameterizations– Since they wont have to be computed by the developers, a

source for mistakes would be eliminated.

• For the long run, we should consider providing operators for computing divergence and curl of a vector field.– When a new coordinate system is introduced, these

operators would have to be implemented for the new coordinate system.

– Parameterizations or other features resorting to divergence and curl of vector fields would be automatically applicable.

Concluding Remarks

• Some of the initial modularity concepts are broken, intentionally or unintentionally.

• Existing framework is not very supportive of convoluted development

• 15+ years after the initial design, it is time to go back to the drawing board.

• Leading developers should work together:– Find ways to involve the Community in academic

development and vice versa (e.g., joint proposals, visiting scientist programs) .

– Start a developers forum that would instigate technical discussion and create a framework for morphing ideas

– Have periodic developers workshops

Georgia Institute of Technology

Parallelization

• We are moving fast towards an era of computationally intensive applications, e.g. forecasting, variational inversion, forward and adjoint sensitivity analysis, uncertainty analysis, etc.

• CMAQ’s parallelization is inefficient and outdated– Particularly when we have 24-thread personal

computers

• Part of the problem is IOAPI whose parallelization needs revisiting.

• A new vertical coordinate was introduced in CMAQ to make it compatible with WRF

• This coordinate is a function of time• What about apparent fluxes due to movement of

vertical layers?

• As I recall, the governing equations of CMAQ do not have any terms to accommodate a moving grid. They assume a coordinate system that is constant in time such as VGTYPE = 2

VGTYPE = 7

w