JohnnyWei-Bing Lin

A Hands-On Introduction to UsingPython in the Atmospheric and

Oceanic Sciences

http://www.johnny-lin.com/pyintro

2012

c© 2012 Johnny Wei-Bing Lin.Some rights reserved. Printed version: ISBN 978-1-300-07616-2. PDF ver-sions: No ISBNs are assigned.

This work is licensed under the Creative Commons Attribution-Noncom-mercial-Share Alike 3.0 United States License (CC BY-NC-SA). To view acopy of this license, visit http://creativecommons.org/licenses/by-nc-sa/3.0/

us or send a letter to Creative Commons, 171 Second Street, Suite 300, SanFrancisco, California, 94105, USA.

Who would not want to pay money for this book?: if you do not needa black-and-white paper copy of the book, a color PDF copy with functionalhyperlinks, have limited funds, or are interested in such a small portion ofthe book that it makes no sense to buy the whole thing. The book’s web site(http://www.johnny-lin.com/pyintro) has available, for free, PDFs of everychapter as separate files.

Who would want to pay money for this book?: if you want a black-and-white paper copy of the book, a color PDF copy with functional hy-perlinks, or you want to help support the author financially. You can buya black-and-white paper copy of the book at http://www.johnny-lin.com/

pyintro/buypaper.shtml and a hyperlink-enabled color PDF copy of the bookat http://www.johnny-lin.com/pyintro/buypdf.shtml.

A special appeal to instructors: Instruction at for-profit institutions, asa commercial use, is not covered under the terms of the CC BY-NC-SA, andso instructors at those institutions should not make copies of the book forstudents beyond copying permitted under Fair Use. Instruction at not-for-profit institutions is not a commercial use, so instructors may legally makecopies of this book for the students in their classes, under the terms of the CCBY-NC-SA, so long as no profit is made through the copy and sale (or FairUse is not exceeded). However, most instruction at not-for-profit institutionsstill involves payment of tuition: lots of people are getting paid for theircontributions. Please consider also paying the author of this book somethingfor his contribution.

Regardless of whether or not you paid money for your copy of the book,you are free to use any and all parts of the book under the terms of the CCBY-NC-SA.

Glossary

attribute data bound to an object that are designed to be acted on by meth-ods also bound to that object.

calling execute or run a function.

class the template or “pattern” all instances of that class follow.

data coordinates a coordinate system for a plot where locations are speci-fied by the values of the x- and y-axes data ranges.

delimit show where a sequence or collection begins and ends.

development environment an application that facilitates software develop-ment, often by providing coding, documentation, debugging, and exe-cution tools in one place.

docstring a triple-quote delimited string that goes right after the def state-ment (or similar construct) and which provides a “help”-like descrip-tion of the function.

dynamically typed variables take on the type of whatever value they are setto when they are assigned.

exception an error state in the program that cannot be processed by the cur-rent scope.

immutable a variable/object that cannot be changed.

import compile a module or package and make what is in the module orpackage accessible to the Python program that is doing the importing.

inherit incorporate the attribute and method definitions of another class intoa definition of a new class of objects.

171

Glossary

inheritance dealing with inheriting attribute and method definitions of an-other class into a definition of a new class of objects.

instance an object that is the specific realization of a class of objects.

instantiate create an instance of a class.

instantiating creating an instance of a class.

instantiation the act of creating an instance of a class.

interpreter the execution environment for Python commands.

iterable a data structure that one can go through, one element at a time; insuch a structure, after you’ve looked at one element of it, it will moveyou on to the next element.

iterator used nearly interchangably with the noun form of “iterable”.

method functions bound to an object that are designed to act on the dataalso bound to that object.

module an importable Python source code file that typically contains func-tion, class, and variable object definitions.

multi-paradigm language a computer language that supports multiple pro-gramming methodologies, for instance, object-oriented programmingand procedural programming.

mutable a variable/object that can be changed.

namespace a set of function, variable, class, etc. names; these names can bestored inside an object variable and referenced via that variable.

newline character a special text code that specifies a new line; the specificcode is operating system dependent.

object a “variable” that has attached to it both data (attributes) and functionsdesigned to act on that data (methods).

object file for a compiled language, this is a file produced by the compilerafter compiling the source code file; this is not an object in the senseof object-oriented programming.

172

Glossary

package a directory of importable Python source code files (and, potentially,subpackages) that typically contains function, class, and variable ob-ject definitions.

package manager a program that streamlines the installation of tools andapplications as part of an operating system or distribution; this is notto be confused with a Python package, which is not, in general, anoperating system or distribution package.

procedural programming a programming paradigm where a program isbroken up into discrete procedures or subroutines, each of which doa specified task and communicate with the rest of the program solely(ideally) through input and output variables that are passed in argu-ment lists and/or return values..

PyAOS a web community whose goal is to support the use of Python in theatmospheric and oceanic sciences; see http://pyaos.johnny-lin.com.

rank the number of dimensions in an array; thus, a 2-D array has rank 2.

runtime when some code or a program is actually executing.

shape a tuple whose elements are the number of elements in each dimensionof an array; in Python, the elements are arranged so the fastest vary-ing dimension is the last element in the tuple and the slowest varyingdimension is the first element in the tuple.

terminal window a text window in which you can directly type in operatingsystem and other commands.

typecode a single character string that specifies the type of the elements ofa NumPy array.

173

Glossary

174

Acronyms

AMS American Meteorological Society.

AOS atmospheric and oceanic sciences.

API application programming interface.

CDAT Climate Data Analysis Tools.

cdms Climate Data Management System.

CISL Computational Information Systems Laboratory.

dpi dots per inch.

EPD Enthought Python Distribution.

GCM general circulation model.

GUI graphical user interface.

HOPS Hyperslab OPerator Suite.

i/o input/output.

IDL Interactive Data Language.

LLNL Lawrence Livermore National Laboratory.

NCAR National Center for Atmospheric Research.

NGL NCAR Graphics Language.

NRCC Northeast Regional Climate Center.

175

Acronyms

OO object-oriented.

OOP object-oriented programming.

PCMDI Program for Coupled Model Diagnostics and Intercomparison.

UV-CDAT Ultrascale Visualization-Climate Data Analysis Tools.

vcs Visualization Control System.

176

Bibliography

Basili, V. R. and Selby, R. W. (1987). Comparing the effectiveness of soft-ware testing strategies. IEEE Trans. Software Eng., SE-13(12):1278–1296.

Curtis, B. (1995). Objects of our desire: Empirical research on object-oriented development. Human-Computer Interaction, 10:337–344.

Lin, J. W.-B. (2009). qtcm 0.1.2: a Python implementation of the Neelin-Zeng Quasi-Equilibrium Tropical Circulation Model. Geosci. Model Dev.,2:1–11, doi:10.5194/gmd–2–1–2009.

Lin, J. W.-B. (2012). Why Python is the next wave in earth sciences com-puting. Bull. Amer. Meteor. Soc., (submitted).

Martelli, A. (2006). Python in a Nutshell. O’Reilly Media, Sebastopol, CA,2nd edition.

177

BIBLIOGRAPHY

178

Index

allclose, 19append, 24arange, 49, 50ArcGIS, 167arguments, see parametersarray, 40, 48arrays, 47

array syntax, 59, 60boolean, 65comparisons, 59, 64–71compatibility checking, 60converting types, 55creating, 47, 50, 55data types, 48, 53, 55element ordering, 51flexible code, 54, 64help, 72indices, 50inquiry, 53line continuation, 52looping through, 58loops vs. array syntax, 59multi-dimensional, 51operations, 58, 60, 69operators as functions, 60rank, 53, 60reshape, 54shape, 53size, 53, 54slicing, see slicing, 84subarrays, 53typecodes, see arrays, data types

assignment, 17, 74, 132, 140

dictionary elements, 95list elements, 23, 26reference vs. value, 140using dictionaries for, 93

assignValue, 84astype, 55, 77, 91, 102attrgetter, 111attributes, 41, 98, 138

delete, 133get, 133inquiry, 133listing, 42private, 101public, 102setting, 133

axis, 158

backslashline continuation, 26string character, 19

barbs, 156Basemap, 158

coastlines, 160contour plots on a map, 159cylindrical projection, 160installing, 159latitude lines, 160longitude lines, 160

boxfill, 124

calculator, 14call , 160

Callahan, Steven, 5CapWords, 105

179

INDEX

case sensitivity, 18CDAT, 78, 80, 167cdms2, 124clabel, 155, 157clarity, 2class, 98, 104close, 74cm, 155cmap, 155colons, 34colorbar, 155, 158colors, 155command history, 12, 13comment character, 62commercial software, 7common blocks, 118concatenate, 55continuation character, see backslash,

line continuationcontour, 154, 158contour plots, see matplotlib, contour

plotscontourf, 155, 158copy, 140correlate, 71count, 100course files, viii, 9createDimension, 84createVariable, 84, 85Ctrl-d, 10cumsum, 103

data analysis, 89dynamic, 131missing values, 121

deepcopy, 140def, 29, 63, 104delattr, 132delimiting code blocks, 30development environments, 11dict , 137

dictionaries, 26, 93, 94, 137dynamically filling, 95flexible code, 95, 134keys, 27, 29methods, 27values, 27

dir, 11, 42, 99directory listing, 93doc , 101

docstrings, see documenting codedocumenting code, 62, 166

docstrings, 63Doutriaux, Charles, 124, 125Drach, Bob, 124, 125dtype, 48, 53, 103dynamically typed, see types, dynamic

elif, 34else, 34Enthought Python Distribution, 8Epoch, 70Epydoc, 166except, 44exceptions

exception classes, 43, 45handling, 44, 165throwing, 43

exp, 71

f2py, 166False, 20fft, 71figure, 150, 152file input/output, 90

close file objects, 74file objects, 74multiple-column text, 79netCDF, see netCDFopen to append, 74open to read, 74open to write, 74

180

INDEX

reading a text file, 75single-column text, 77writing to a text file, 75

filled, 128, 129fill value, 123, 128Fiorino, Michael, 5float, 76, 78fontsize, 155for, 34free gift, ixfunctional programming, 1functions, 29, 138

as objects, 94calling, 138parameters, see parametersreturn values, 29, 62

getattr, 132, 138getValue, 81glob, 93GNU/Linux, 8, 9GRIB, 87

hamming, 71hasattr, 132has key, 28HDF, 87hello world, 10, 12help, 11, 72histogram, 71Hunter, John, 144

id, 141IDL to Python, 168IDLE, 12if, 33, 64import, 39importing

aliasing, 41data, 41functions, 41

indentation, 29

inheritance, 106, 165, 166init , 104, 106, 111insert, 24installing, 7int, 76, 95interp, 71interpreter, 10–11

exit, 10, 12IPython, 11is, 21isupper, 100

join, 76

keys, 28kurtosis, 96

len, 22, 38levels, 154, 155line plots, see matplotlib, line plotslinesep, 77Linux, see GNU/Linuxlists, 22, 137

complex references, 23indices, 22, 23initialize, 38lengths, 22looping through, 34methods, 24shuffling, 139slicing, see slicing

logical testing, 33compound tests, 33

logical and, 65logical not, 69logical or, 65looping, 34

by indices, 35iterators, 35

ma, 40, 126Mac OS X, 9

181

INDEX

main , 112map projections, see Basemapmasked arrays, 40, 122, 126–130

converting to an array, 128creating, 126, 127fill values, 123, 128masks, 123, 129operations, 123, 130

masked variables, 122, 124creating, 126

masked array, 126masked greater, 127masked where, 127Matlab to Python, 168matplotlib, 143

axis labeling, 153Basemap, see Basemapcolor bars, 155color maps, 155colors, 145, 149contour levels, 154contour plots, 154contour plots on a map, 159displaying vs. saving figures, 152filled contour plots, 155line and marker property listings,

146line plots, 144lined and filled contour plot, 155linestyle, 145, 147linewidth, 145map projections, see Basemapmarkers, 145, 148multiple curves on one figure, 151multiple independent figures, 150negative contours dashed, 155pyplot, 144save figure, 152, 154save figure then visualize, 154

save figure without displaying, 147,154

using LATEX to annotate plots, 146visualizing plots, 144wind barbs, 156

max, 42mean, 90median, 90meshgrid, 56, 117, 156methods, 41, 98, 99, 138

calling, 100, 102defining, 104, 109delete, 133get, 133inquiry, 133listing, 42private, 101public, 102setting, 133

min, 42missing values, see data analysis, miss-

ing values; masked arraysmodeling, 137, 141modules, 39

importing, 39, 40submodules, 40

name , 112namespaces, 2, 40

module names vs. namespaces, 41preventing collisions, 41, 94

netCDFcreating dimensions, 84creating variable objects, 84dimensions, 80, 81file objects, 81filling array variables, 84filling scalar variables, 84global attributes, 80, 83metadata, 82reading a variable, 81

182

INDEX

structure, 80unlimited dimension, 83variables, 80, 81

newline character, 19, 75, 77, 78nlevels, 154None, 21Noon, William, 6NumPy, see also arrays, 40, 47, 126

importing, 47, 49, 126

object, 106object-oriented programming, 97–99

vs. procedural, 113, 115, 119, 120,137

objects, 110attributes, see attributescalling, 160classes, 98, 104, 110inheritance, see inheritanceinstances, 98, 106, 110instantiation, 104, 116, 134listing attributes and methods, 42,

99methods, see methodsprogramming, see object-oriented

programmingsyntax, 41, 100

open, 74, 90OpenDAP, 167operators

addition, 18defining, 101division, 15, 18, 19equal, 18, 21exponentiation, 18greater than, 18greater than or equal to, 18is, 21less than, 18less than or equal to, 18logical, 20

multiplication, 18not equal, 18subtraction, 18

ordinal value, 22orientation, 155os, 77, 166

paths, 166

package manager, 8packages, see modulespandas, 167parameters

functions, 29, 30initialize, 22, 31, 134keyword, 30passing in lists of arguments, 32positional, 30

ParaView, 162pass, 64permutations, 139platform independence, 1, 77, 166plot, 144potential temperature, 62print, 14, 19, 102procedural programming, 98

vs. object-oriented, 113, 115, 119,120

programmingdynamic subroutine management,

137dynamic variable management, 131,

133provenance management, 3PyAOS, 169PyGrADS, 162, 167PyNGL, 78, 143, 162, 167PyNIO, 80, 87, 167pyplot, see matplotlib, pyplotpysclint, 80, 87PyTables, 73, 80, 87pytest, 166

183

INDEX

Python(x,y), 11PYTHONPATH, 41PyUnit, see unittest

raise, 43range, 35rank, 53ravel, 55, 103readline, 75readlines, 75, 79reference manuals, 168remove, 24repeat, 55reshape, 54, 103resize, 103reStructuredText, 63return, 29, 62reverse, 42round, 103RPy, 167runlist, 137

SAGE, 167Saravanan, R., 136savefig, 152, 154, 158ScientificPython, 80

importing, 80SciPy, 160, 165, 167

importing, 165scripting, 1self, 104, 107, 110setattr, 107, 132shape, 53, 102show, 144, 158sin, 40, 71size, 53skew, 96slicing

arrays, 50, 53lists, 23strings, 25

sorted, 28, 111, 112sorting, 93, 112Sphinx, 63, 166split, 76Spyder, 11std, 90strings, 19

attributes, 99concatenation, 20, 76, 114converting, 76methods, 99splitting, 76triple quotes, 20

style guide, 46subplot, 156sys, 166

search path, 166

T, 102, 103tab character, 19, 76, 79terminal window, 11testing, 112, 166time, 70timings, 70title, 99Tk, 12transpose, 103transpose, 55, 102True, 20try, 44tutorials, 168typecodes, see arrays, data typestypes

arrays, see arraysbasic, 17booleans, 20dictionaries, see dictionariesdynamic, 17, 22, 35, 92floating, 19integers, 19lists, see lists

184

INDEX

NoneType, 21strings, see stringstuples, 25upcasting, 19

underscore, see attributes, private; meth-ods, private

unittest, 112, 166upper, 99, 100UV-CDAT, see also CDAT; cdms2, 80,

87, 124, 162, 167

ValueError, 43, 44values, 28vcs, 124, 143VisTrails, 162visualization, 143, 162VPython, 163

weather maps, 3where, 66, 67while, 36widgets, 12Williams, Dean, 124, 125Windows, 8write, 75writelines, 75WxMAP2, 3

xrange, 59

zeros, 49

185

INDEX

186

About the Author

Johnny Wei-Bing Lin graduated from Stanford Universitywith a B.S. in Mechanical Engineering and an M.S. in CivilEngineering-Water Resources. After working as an environ-mental engineer, he returned to school and received his Ph.D.in Atmospheric Sciences from UCLA. His atmospheric sci-ences research is focused on stochastic convective param-eterizations, ice-atmosphere interactions in the Arctic, andsimple frameworks for modularizing climate models. Hehas chaired the AMS Python Symposiums and has taught orco-taught some of the AMS Python short courses. Johnnyalso helps coordinate the PyAOS mailing list and blog (http://pyaos.johnny-lin.com), an effort at building up the atmo-spheric and oceanic sciences Python community. Currently,he is a Professor of Physics at North Park University inChicago.

Colophon

This book was written using PDFLATEX (PDFTEX 3.1415926-1.40.10-2.2) and the Vim editor, running on an Ubuntu 12.04GNU/Linux system. Times-like fonts are provided by the TXFonts package (http://www.ctan.org/pkg/txfonts). The titlepage and examples environment are based on the “titleTMB”example from Peter Wilson’s July 13, 2010 work Some Ex-amples of Title Pages (http://www.ctan.org/tex-archive/info/

latex-samples/TitlePages).