Embed Size (px)
DESCRIPTION
this script is given to students of physics lectures
Citation preview
© Gerhard Brunthaler - 1 -
Python for Lectures
V05 – Jan 2011
2010/11
Content 1 Installation ...........................................................................................................................3 2 First Trials............................................................................................................................3
2.1 Pylab and IPython .......................................................................................................3 2.2 IDLE............................................................................................................................4 2.3 SciTe Editor ................................................................................................................5
3 Session 2.............................................................................................................................6 3.1 Short programs ...........................................................................................................6 3.2 Tutorials ......................................................................................................................8
4 Modules, Plotting, etc. .........................................................................................................8 4.1 Modules ......................................................................................................................8 4.2 Plotting........................................................................................................................9
5 Statements, control flow, indentation, advantage...............................................................13 5.1 for loop......................................................................................................................14 5.2 Indentation ................................................................................................................14 5.3 if – elif – else .............................................................................................................14 5.4 while..........................................................................................................................15 5.5 try..............................................................................................................................15 5.6 Advantages of Python: ..............................................................................................16
6 Numbers, Strings, Boolean................................................................................................16 6.1 Everything is an Object .............................................................................................16 6.2 Number formats ........................................................................................................16 6.3 Strings.......................................................................................................................18 6.4 Truth Value, Boolean Operations ..............................................................................18 6.5 Declaring variables....................................................................................................19
7 Datatypes ..........................................................................................................................20 7.1 Lists ..........................................................................................................................20 7.2 Tuples .......................................................................................................................21 7.3 Dictionaries ...............................................................................................................22
8 Functions...........................................................................................................................23 8.1 One return value .......................................................................................................23 8.2 No return value..........................................................................................................24 8.3 Multiple return values ................................................................................................24 8.4 Default argument values ...........................................................................................25 8.5 Lambda expressions .................................................................................................26 8.6 Built-in functions........................................................................................................26
9 Important built-in modules .................................................................................................27 9.1 The sys module.........................................................................................................28 9.2 The os module ..........................................................................................................29
10 Pitfalls ...........................................................................................................................31 10.1 The object pitfall ........................................................................................................31 10.2 More pitfalls...............................................................................................................33
11 Introspection .................................................................................................................33 12 Appendix.......................................................................................................................33
Python for Lectures
- 2 -
12.1 Conversion of a Matlab file into a Python file.............................................................34 12.2 Figures with matplotlib...............................................................................................34 12.3 Nonblocking Plots with Matplotlib ..............................................................................35 12.4 Matplotlib's OO Class Library ....................................................................................36 12.5 File and Directory handling/manipulation...................................................................36 12.6 Measuring raw speed................................................................................................37 12.7 Video processing.......................................................................................................37
13 Other stuff .....................................................................................................................39 13.1 Unordered Stuff.........................................................................................................39
14 Version changes ...........................................................................................................41
Foreword
I have heard from several students that it is difficult to find a programming language which can be used to solve exercises in physics, for plotting scientific diagrams and for simulations. It seems to me that Python is very suitable to perform this tasks. An important feature is also that it is a GPL -compatible language (GPL = General Public License). So it can be used free of charge by students and any other persons in contrast to packages like Matlab, which I use a lot for my scientific work at the university institute.
Python is a general purpose programming language. It is interpreted and dynamically typed and is very suited for interactive work and quick prototyping, while being powerful enough to write large applications in.
Note: On my computer, I use the English version of the Windows operating system as well as all software in English versions if they are available. I am also used to describe and comment my scientific work and software stuff in simple English language and thus I go on in that way here.
Python for Lectures
- 3 -
1 Installation
I suggest to use Enthought Python for scientific programming work. Enthought is a company which offers a programming package around python. It is free for personal and academic use, but companies have to pay for it. Before that, I installed pure Python and then added packages to it. It took me days to get a useful system up. The Enthought package works at once and you can indeed start to work on Python itself.
Download from end of page: http://www.enthought.com/products/getepd.php , free use for academic and students!
Installation on Windows XP. I create usually a system restore point before larger installations or changes, in order to be able to set my computer back to a state before that procedure.
Click installer, choose installation folder …, choose to put icons onto desktop and install!
2 First Trials
After installation, there are two new Icons on the tesktop: PyLab and Mayavi. PyLab opens the an interactive shell “IPython”, whereas Mayavi is an 3D scientific data visualization and plotting package. First we have a look on PyLab.
2.1 Pylab and IPython
Clicking on “Pylab” opens up a black command window and after 30 seconds or a minute (be patient) some text appears:
********************************************************************** Welcome to IPython. I will try to create a personal configuration directory where you can customize many aspects of IPython's functionality in: C:\Documents and Settings\bru\_ipython Initializing from configuration: C:\Prog\Python26EPD\lib\site-packages\IPython\ serConfig Successful installation! Please read the sections 'Initial Configuration' and 'Quick Tips' in the IPython manual (there are both HTML and PDF versions supplied with the distribution) to make sure that your system environment is properly configured to take advantage of IPython's features. Important note: the configuration system has changed! The old system is still in place, but its setting may be partly overridden by the settings in "~/.ipython/ipy_user_conf.py" config file. Please take a look at the file if some of the new settings bother you. Please press <RETURN> to start IPython.
Python for Lectures
- 4 -
IPython is an interactive shell for the Python programming language.
After pressing return in the command window, one gets further:
********************************************************************** Enthought Python Distribution -- http://www.enthought.com Python 2.6.6 |EPD 6.3-2 (32-bit)| (r266:84292, Sep 20 2010, 11:26:16) [MSC v.150 0 32 bit (Intel)] Type "copyright", "credits" or "license" for more information. IPython 0.10.1 -- An enhanced Interactive Python. ? -> Introduction and overview of IPython's features. %quickref -> Quick reference. help -> Python's own help system. object? -> Details about 'object'. ?object also works, ?? prints more. Welcome to pylab, a matplotlib-based Python environment. For more information, type 'help(pylab)'. In [1]:
type: “exit()” in order to finish this IPython shell.
The interactive shell IPython is a component of the SciPy package.
For more info see e.g. http://en.wikipedia.org/wiki/IPython or see a short video on the use of it http://showmedo.com/videotutorials/series?name=PythonIPythonSeries .
IPython can be used to perform simple one-line statements and calculations, but also to edit and run large Python programs.
2.2 IDLE
IDLE is the simplest integrated development environment (IDE) for Python, which is bundled in each release of the programming language since version 2.3.
In the Windows “Programs” folder in the start menue, the new folder “Enthought” has appeared. Inside there are icons for “Documentation”, “Python Manuals”, “IDLE” and others. Start IDLE by double-clicking the corresponding icon. You should get a window (with white background color?) with the following text:
Python 2.6.6 |EPD 6.3-2 (32-bit)| (r266:84292, Sep 20 2010, 11:26:16) [MSC v.1500 32 bit (Intel)] on win32 Type "copyright", "credits" or "license()" for more information. **************************************************************** Personal firewall software may warn about the connection IDLE makes to its subprocess using this computer's internal loopback interface. This connection is not visible on any external interface and no data is sent to or received from the Internet. **************************************************************** IDLE 2.6.6 >>>
As a first step in the Interactive Shell type
Python for Lectures
- 5 -
>>> 1 + 1
by pressing return at the end of the line, you should get
2
and further
>>> x = 1 >>> y = 2 >>> x + y 3
As you see, IDLE can be used for simple calculations, but also for more advanced work.
Close IDLE by entering “exit()” into the next line, or by clicking into the upper right closing button of the active Windows window.
I personally find IDLE and IPython not very convenient environments to work with, so we try another one included in the Enthought distribution package.
2.3 SciTe Editor
SciTE or SCIntilla based Text Editor is a cross-platform text editor http://en.wikipedia.org/wiki/SciTE .
In the “Documentation” folder in the Windows “Programs” “Enthought” folder you find at position 88 the link http://www.scintilla.org/SciTEDoc.html to the SciTe documentation for detailed information!
The icon for the SciTe editor is also inside the “Enthought” folder in the Windows “Programs” folder in the start menue. Start SciTe and type:
print “Hi”
into the first line of the editor, save the file as “printhi.py” to a convenient folder; the file extension is important in order that the editor recognizes the text as Python code and performs a corresponding text highlighting after saving.
Note: When working on a Linux system, it is usually necessary to include: #!/usr/bin/python into the first line, so that the Python interpreter is addressed. It is not clear to me, whether this is also necessary when working with the SciTe editor.
Press F5-key (or go to Menue “Tools” – “Go”) to execute the python code!
An attached window opens up and prints the output as:
>pythonw -u "printhi.py" Hi >Exit code: 0
Congratulations, you have run your first Python script!
“Exit code: 0” means that there was no error.
Python for Lectures
- 6 -
3 Session 2
3.1 Short programs
Here is the next small script file (program) to demonstrate a little more on the “print” statement and on commenting with the hash sign “#”.
# prog_01.py # calculation and window closing # everything after the hash character '#' is considered as comment # [email protected] # printing into output window: print "Hi" print 1 + 2 print " " # create empty line # do not type any space in front of line - this would cause an error! # if one works within an Editor, IDE or so, the output window stays open # and you can see the result. But if you run the python script by double- # clicking the file in the folder, under Windows the python interpreter # (if correctly installed) may be called directly to perform the script. # In that case it closes the output window immediately after finishing # and you cannot read the result. # In such a case you can add e.g. an 'raw_input' command in order # that the python interpreter waits for an input before finishing. # For activation, remove the two leading hash-characters from next line: ##raw_input("Press return to close this window ... ")
I could also send the script files separately, but I think that this would have a smaller learning effect in that the participants are probably less active.
# prog_02.py # simple calculation and formated output # [email protected] a = 2 b = 3 c = a*b print c print " " # empty line print "a*b = ", c print " " print "formated output:" print "a = %g, b = %g gives c = a*b = %g" % (a,b,c) # for %-formating see "5.6.2. String Formatting Operations" in # http://docs.python.org/library/stdtypes.html#typesseq-strings
String Formatting Operations: see 5.6.2. in: http://docs.python.org/library/stdtypes.html#typesseq-strings
Conversion Meaning Notes 'd' Signed integer decimal. 'i' Signed integer decimal.
Python for Lectures
- 7 -
'o' Signed octal value. (1) 'u' Obsolete type – it is identical to 'd'. (7) 'x' Signed hexadecimal (lowercase). (2) 'X' Signed hexadecimal (uppercase). (2) 'e' Floating point exponential format (lowercase). (3) 'E' Floating point exponential format (uppercase). (3) 'f' Floating point decimal format. (3) 'F' Floating point decimal format. (3) 'g' Floating point format. Uses lowercase exponential format if exponent is less than -4 or not less than precision, decimal format otherwise. (4) 'G' Floating point format. Uses uppercase exponential format if exponent is less than -4 or not less than precision, decimal format otherwise. (4) 'c' Single character (accepts integer or single character string). 'r' String (converts any Python object using repr()). (5) 's' String (converts any Python object using str()). (6) '%' No argument is converted, results in a '%' character in the result. For Notes see explanation in above given link.
next program example:
# prog_03.py # more simple calculations # [email protected] # printing into output window: print "Simple calculations:" print " 2/3 = ", 2/3, "- here an integer division has been performed!" print " 2.0/3 = ", 2.0/3, "- here a floating point operation has been performed!" print " (1+2j)*(2+3j) =", (1+2j)*(2+3j), "- complex numbers" # but functions like sin(x), sqrt(x) are not defined in the base module # you have to import modules! # the next statement leads to an error message: sin(0.5) # also the constant 'pi' is not yet defined
In order to work with mathematical function one needs to add a module to the script. This is shown in the next example.
# prog_04.py # more calculations, module import # [email protected] # importing a mathematic module: from pylab import * # include function e.g. as sin(0) print "Functions from mudule:" print " sqrt(2) = ", sqrt(2) print " pi = ", pi print " sin(pi/2) = ", sin(pi/2) print " exp(1j*pi/2) = ", exp(1j*pi/2), " - equal to 0+1j within rounding errors"
With such modules Python programming gets a little more messy, but only with those one can do powerful work.
Python for Lectures
- 8 -
3.2 Tutorials
For those who can’t wait for any longer, just go by yourself for one of the many tutorials and introductions on Python like http://docs.python.org/tutorial/introduction.html or http://diveintopython.org/.
A good introduction into Python can be found in the online tutorial http://www.network-theory.co.uk/docs/pytut/index.html by Guido van Rossum, the creator of the Python language himself. This tutorial can also be ordered as a book.
4 Modules, Plotting, etc.
4.1 Modules
Working with modules – see next example.
#!/usr/bin/python # prog_05_module_1.py # works with Python 2.6.2 # the first line above is necessary if working on a Linux system! import pylab # include function e.g. as pylab.sin(0) # module pylab is a part of module 'matplotlib' print "sin(pi/6) =", pylab.sin(pylab.pi/6) # work with complex numbers - be careful: print "square-root of -2 =", pylab.sqrt(-2) # gives "not a number"-output! print "square-root of -2+0j =", pylab.sqrt(-2+0j) # gives complex number
With
import pylab
the module named ‘pylab’ gets linked to the program. It includes many mathematical functions. If a module is included in that way, all functions (“subroutines”) of that module have to be called by giving the module name together with the function name, e.g. ‘pylab.sin’ or ‘pylab.pi’. This is cumbersome, but ensures that even if two functions inside two different modules have the same name, they will not be mixed up.
Nevertheless, in most cases one likes to have a compact naming for functions. In that sense it is possible to use
from pylab import *
which means that all function (because of ‘*’) from the module are imported directly with their names without any prename, e.g. directly ‘sin’ or ‘pi’. See the following example.
# prog_06_module_2.py from pylab import * # include function e.g. as sin(0) print "sin(pi/6) =", sin(pi/6) # work with complex numbers - be careful: print "square-root of -2+0j =", sqrt(-2+0j) # gives complex number print "exp(1) =", exp(1)
Python for Lectures
- 9 -
Instead of importing all functions from the module, one can import a certain set by listing them up, as in:
# prog_06b_module_2b.py from pylab import pi,sin,exp # include function e.g. as sin(0) print "sin(pi/6) =", sin(pi/6) print "exp(1) =", exp(1)
A compromise between the options above is to use the import statement as in:
# prog_07_module_3.py import pylab as p # include function e.g. as p.sin(0) pi = p.pi # here a new name is assigned to p.pi print "sin(pi/6) =", p.sin(pi/6) print "exp(1) =", p.exp(1)
Here, to all pylab functions the prename ‘p’ is assigned to have a short form, but still to reflect from which module the functions are coming. With that form of import, “one and the same” function can be important from different modules, like in:
# prog_07b_module_3b.py import pylab as p # include function e.g. as p.sqrt (0) import numpy.lib.scimath as s # include function e.g. as s.sqrt (0) # work with complex numbers - be careful: print "pylab square-root of -2 =", p.sqrt(-2) # nan (not a number) print "scimath square-root of -2 =", s.sqrt(-2) # complex number
The last example shows that the square root function for negative numbers behaves differently in different modules, as these modules were written by different persons or groups and for different purposes. This has disadvantages and makes a careful handling of functions from different modules very important.
4.2 Plotting
There are different possibilities for plotting under Python. The most promising one is presently “Matplotlib”. It allows calculations and plotting in a very similar way as with Matlab. If you work with the Enthought Python package, matplotlib should be ready for use. If not, you have to download matplotlib from http://matplotlib.sourceforge.net/ and install it according to the documentation there (I did this some time ago, but for now I work with the Enthought package).
Let’s start with a simple program to demonstrate the simplicity of plotting:
# prog_09_plotting_1.py # Python version and matplotlib version have to match in order to work properly! import matplotlib matplotlib.use('TkAgg') # define backend before pylab import! from pylab import * # include function e.g. as sin(0) x = linspace(0,2*pi,50) # defines a number array from 0 to 2*pi with 50 points y = sin(x) # calculate the sin for all individual elements of the array
Python for Lectures
- 10 -
figure(1) # num=1 plot(x,y) xlabel('x') ylabel('sin(x)') title('Example Plot 1') show()
A separate window with the plot of a sin-curve should open up if everything is installed correctly. If that is the case, please stare for a while onto the image in order to appreciate your success! It should look the image, which is partly shown below.
Fig.: Cut-out of screenshot of image produced by “prog_09_plotting_1.py”. In order to see the full image, run the program by yourself!
If the separate window has not opened up, don’t worry and solve the problems by googleing for the matplotlib installation problems and start a discussion in our work group!
In order that you are able to continue your work with Python, you have first to close the separate plot window (at least on my Win-XP installation with Enthought Python and SciTe).
Now let’s consider some details of the modules used above.
Here some sentences from http://en.wikipedia.org/wiki/Matplotlib:
Matplotlib is a plotting library for the Python programming language and its NumPy numerical mathematics extension. It provides an object-oriented API which allows plots to be embedded into applications using generic GUI (i.e. graphical user interface) toolkits, like wxPython, Qt, or GTK. There is also a procedural "pylab" interface based on a state machine (like OpenGL), designed to closely resemble that of MATLAB. Matplotlib is written and maintained primarily by John Hunter, and is distributed under a BSD-style license. The package pylab combines pyplot with numpy into a single namespace.
Comparison with MATLAB
The pylab interface makes Matplotlib easy to learn for experienced MATLAB users, resulting in a viable alternative for many MATLAB users as a teaching tool for numerical mathematics and signal processing.
Some of the advantages of Python+NumPy+Matplotlib over MATLAB include: * Based on Python, a full-featured modern object-oriented programming language suitable for large-scale software development * Suitable for fast scripting, including CGI scripts * Free, open source, no license servers * Native SVG support
(i.e. http://en.wikipedia.org/wiki/SVG: SVG means “Scalable Vector Graphics”.)
http://en.wikipedia.org/wiki/NumPy:
NumPy is an extension to the Python programming language, adding support for large, multi-dimensional arrays and matrices, along with a large library of high-level mathematical functions to operate on these arrays. The ancestor of NumPy, Numeric, was originally created by Jim Hugunin. NumPy is open source and has many contributors.
Python for Lectures
- 11 -
Since the standard Python implementation is an interpreter, mathematical algorithms often run much slower than compiled equivalents, such as those written in C. NumPy addresses this problem for many numerical algorithms by providing multidimensional arrays and lots of functions and operators that operate on arrays. Thus any algorithm that can be expressed primarily as operations on arrays and matrices can run almost as fast as the equivalent C code.
This means that “pyplot” is a module which includes the plotting capabilities, “numpy” adds the capability to work with number arrays and matrices. Normally numpy is a separate module.
Pylab is a part of matplotlib and combines pyplot and numpy into one module with a common namespace. Normally each module has its separate namespace, i.e. it has its own memory for function names and variables, which can not be seen directly by the calling program. But with modulename.function (like pylab.sqrt) one has access from the calling program to the module namespace. In that Pylab combines pyplot and numpy, the two modules work hand in hand in a common namespace without any barrier in between.
Using matplotlib in a python shell: http://matplotlib.sourceforge.net/users/shell.html:
By default, matplotlib defers drawing until the end of the script because drawing can be an expensive operation, and you may not want to update the plot every time a single property is changed, only once after all the properties have changed.
But when working from the python shell, you usually do want to update the plot with every command, eg, after changing the xlabel(), or the marker style of a line. While this is simple in concept, in practice it can be tricky, because matplotlib is a graphical user interface application under the hood, and there are some tricks to make the applications work right in a python shell.
…
What is a backend: http://matplotlib.sourceforge.net/faq/installing_faq.html#what-is-a-backend:
A lot of documentation on the website and in the mailing lists refers to the “backend ” and many new users are confused by this term. matplotlib targets many different use cases and output formats. Some people use matplotlib interactively from the python shell and have plotting windows pop up when they type commands. Some people embed matplotlib into graphical user interfaces like wxpython or pygtk to build rich applications. Others use matplotlib in batch scripts to generate postscript images from some numerical simulations, and still others in web application servers to dynamically serve up graphs.
To support all of these use cases, matplotlib can target different outputs, and each of these capabililities is called a backend; the “frontend” is the user facing code, ie the plotting code, whereas the “backend” does all the dirty work behind the scenes to make the figure. There are two types of backends: user interface backends (for use in pygtk, wxpython, tkinter, qt, macosx, or fltk) and hardcopy backends to make image files (PNG, SVG, PDF, PS).
There are a two primary ways to configure your backend. One is to set the backend parameter in you matplotlibrc file (see Customizing matplotlib): backend : WXAgg # use wxpython with antigrain (agg) rendering
The other is to use the matplotlib use() directive:
import matplotlib matplotlib.use('PS') # generate postscript output by default
Python for Lectures
- 12 -
If you use the use directive, this must be done before importing matplotlib.pyplot or matplotlib.pylab.
If you are unsure what to do, and just want to get cranking, just set your backend to TkAgg. This will do the right thing for 95% of the users. It gives you the option of running your scripts in batch or working interactively from the python shell, with the least amount of hassles, and is smart enough to do the right thing when you ask for postscript, or pdf, or other image formats.
…
Here is a somewhat more advanced version of the plotting example from above:
# prog_10_plotting_2.py # Python version and matplotlib version have to match in order to work properly! import matplotlib matplotlib.use('TkAgg') # define backend before pylab import! # works also if the two lines above are absent!! - some default settings then?? from pylab import * # include function e.g. as sin(0) help(linspace) # gives help on 'linspace' x = linspace(0,2*pi,50) # defines a number array from 0 to 2*pi with 50 points y = sin(x) # calculate the sin for all individual elements of the array figure(1,(10,7.5),90) # num=1, figsize=(8,6), dpi=90 clf(), hold(True) plt1 = plot(x,y,'*-', linewidth=3) # linewidth = 2 setp(plt1, 'color', 'r', 'linewidth', 2.0) # set plot attributes like matlab setp(plt1, 'markersize', 10, 'markerfacecolor', 'g', 'markeredgecolor', 'r') xlabel('x') ylabel('sin(x)') title('Example Plot 2') show()
Fig.: Cut-out of image produced by “prog_10_plotting_2.py”. In order to see the full image, run … !
Example program distributed separately The program “plotting_with_matplotlib_2.py” has been distributed within our online discussion group and shows many more plotting possibilities!
Fig.: Small cut-out of image produced by “plotting_with_matplotlib_2.py” and saved into pdf-file. In order to see … !
See also the Webpage “NumPy for MATLAB users” in order to learn more on NumPy possiblities: http://mathesaurus.sourceforge.net/matlab-numpy.html
Pyplot tutorial: http://matplotlib.sourceforge.net/users/pyplot_tutorial.html
Python for Lectures
- 13 -
Figure Customizing: http://matplotlib.sourceforge.net/users/customizing.html
### FIGURE # See http://matplotlib.sourceforge.net/api/figure_api.html#matplotlib.figure.Figure #figure.figsize : 8, 6 # figure size in inches #figure.dpi : 80 # figure dots per inch #figure.facecolor : 0.75 # figure facecolor; 0.75 is scalar gray #figure.edgecolor : white # figure edgecolor
Plot figure with white background and fill area with plot (no space for axes, etc.), switch axis off (in order not to see any ticks showing in from the borders) and write figtext instead of title:
fig1 = figure(1, figsize=(10.0, 10.0)) # figure with size defined in inch fig1.set_facecolor('w') fig1.subplots_adjust(left=0.0, bottom=0.0, right=1.0, top=1.0) ... axis('off'); grid('off') figtext(0.5, 0.04, t_str, horizontalalignment='center', \ verticalalignment='center', fontsize=16)
Set Figure's position: http://osdir.com/ml/python.matplotlib.general/2004-12/msg00114.html http://osdir.com/ml/python.matplotlib.general/2004-12/msg00115.html http://osdir.com/ml/python.matplotlib.general/2004-12/msg00116.html
For WX backend, in order to set figure position on screen 500 pixel from left and 200 pixel from top, use:
manager = get_current_fig_manager() manager.window.SetPosition((500,200))
In TkAgg it works like this:
get_current_fig_manager().window.wm_geometry("+200+300")
To set the window position to X=200, Y=300.
Create figures for use in article publishing: http://seehuhn.de/blog/59 (see App. 12.2)
How to generate publishable quality images: http://eccentric.mae.cornell.edu/~vh63/matplotlib.html
5 Statements, control flow, indentation, advantage
See e.g.: Python_(programming_language)#Statements_and_control_flow
A Python script consists of a series of statements and comments. Each statement is a command that is recognized by the Python interpreter and executed. Statements are usually one line long, though there are a few ways to make them extend over several lines (http://www.dalkescientific.com/writings/NBN/python_intro/statements.html).
Python's statements include (among others):
• The if statement, which conditionally executes a block of code, along with else and elif (a contraction of else-if).
• The for statement, which iterates over an iterable object, capturing each element to a local variable for use by the attached block.
• The while statement, which executes a block of code as long as its condition is true.
Python for Lectures
- 14 -
Such statements are marked by certain keywords. It turns that Python itself is a remarkably simple programming language – it has a very limited number of keywords.
Python has twenty-eight keywords:
and continue else for import not raise
assert def except from in or return
break del exec global is pass try
class elif finally if lambda print while
5.1 for loop
The ‘for’ statement controls the program flow in the following example. The statement block inside the ‘for’ loop is executed until all given values in the list have been taken. The important thing is that the block inside the ‘for’ loop is not marked by braces or an ‘end’ statement, but it is rather distinguished by indentation.
# prog_11_control_flow_1.py # [email protected] n_sum = 0 for n in [1, 2, 3, 4, 5]: # numbers given in list n_sum = n_sum + n print " last number: ", n # last value of n after leaving loop print " sum from 1 to %g: %g \n" % (n, n_sum) # sum of all
Please run the programs by yourself!
5.2 Indentation
Python uses whitespace indentation , rather than curly braces or keywords, to delimit blocks. An increase in indentation comes after certain statements; a decrease in indentation signifies the end of the current block.
(see: Myths about Indentation: http://www.secnetix.de/olli/Python/block_indentation.hawk )
5.3 if – elif – else
An example with the ‘if-elif-else’ statement:
# prog_11b_control_flow_2.py # the 'if-elif-else statement' n = int(raw_input("Please enter a positive or negative integer: ")) if n<0: print " %s is smaller than zero" % n elif n==0: print " %s is equal to zero" % n else: print " %s is larger than zero" % n
Python for Lectures
- 15 -
5.4 while
The while statement:
# prog_11c_control_flow_3.py # create table of square numbers with 'while'-loop x = 1.0 while x < 6.0: print x, '\t', x**2 # the string '\t' represents a tab character x = x + 1.0
5.5 try
Even if a statement or expression is syntactically correct, it may cause an error when an attempt is made to execute it. Errors detected during execution are called exceptions and are not unconditionally fatal. They can be handled by the “try”-command. See the following example.
def is_number(s): # function defintion try: float(s) return True except ValueError: return False
Although there is a better way to test whether a string can be converted into a number or not. Use the isdigit() function for string objects.
a = "03523" a.isdigit()
True
When to catch exceptions? (http://www.python.org/doc/essays/ppt/hp-training/sld028.htm )
a) When there's an alternative option
try: f = open(".startup") except IOError: f = None # No startup file; use defaults
b) To exit with nice error message
try: f = open("data") except IOError, msg: print "I/O Error:", msg; sys.exit(1)
Hint
Sometimes it is necessary to put an already written program block into a loop (or generally speaking into a control-flow statement) and thus to indent the whole block at once . This can be done by hand, line by line. But it is desirable that a text editor supports that duty directly (this is indeed a matter of the text editor or the IDE and is not an ability of the Python interpreter itself, although it requires it).
Python for Lectures
- 16 -
In SciTe the task can be handled by highlighting the block of lines to be indented and just pressing the Tab-key on the keyboard (at least in windows version SciTe 1.74).
5.6 Advantages of Python:
Here is some motivation, why to go on and learn Python!
Why python? http://www.linuxjournal.com/article/3882
Python as a First Language: http://mcsp.wartburg.edu/zelle/python/python-first.html
6 Numbers, Strings, Boolean
So far we have worked with Python without knowing a lot about the available datatypes.
6.1 Everything is an Object
In Python everything is an object, even a simple letter of number, but you will not notice it until you have quite some experience on it. Probably one isn’t so surprised to hear that a function is an object. Nearly all of that objects have built-in attributes and methods – more on that later.
6.2 Number formats
Python supports 4 types of Numbers, the int, the long, the float and the complex. You don’t have to specify what type of variable you want; Python does that automatically.
• Int: This is the basic integer type in python, it is equivalent to the hardware 'c long' for the platform you are using.
• Long: This is a integer number that's length is non-limited. In python 2.2 and later, Ints are automatically turned into long ints when they overflow.
• Float: This is a binary floating point number. Longs and Ints are automatically converted to floats when a float is used in an expression, and with the true-division // operator.
• Complex: This is a complex number consisting of two floats. Complex literals are written as a + bj where a and b are floating-point numbers denoting the real and imaginary parts respectively.
In general, the number types are automatically 'up cast' in this order:
Int → Long → Float → Complex. The farther to the right you go, the higher the precedence.
See some examples in the IDLE environment (for IDLE see chap. 2.2).
>>> x = 5 >>> type(x) <type 'int'> >>> x = 187687654564658970978909869576453 >>> type(x) <type 'long'> >>> x = 1.34763 >>> type(x) <type 'float'> >>> x = 5 + 2j
Python for Lectures
- 17 -
>>> type(x) <type 'complex'>
However, some expressions may be confusing since in the current version of python, using the / operator on two integers will return another integer, using floor division. For example, 5/2 will give you 2. You have to specify one of the operands as a float to get true division, e.g. 5/2. or 5./2 (the dot specifies you want to work with float) to have 2.5. This behavior is declined and will disappear in a future python release.
>>> 5/2 2 >>> 5/2. 2.5 >>> 5./2 2.5 >>> float(1)/2 # can also be used to give an explicit hint to the pitfall 0.5
See also: http://en.wikibooks.org/wiki/Python_Programming/Numbers
Numbers can also be converted into a certain format by use of built-in constructors int(), long(), float(), and complex(), see e.g.:
>>> a = 1 >>> float(a) 1.0 >>> type(a) <type 'int'>
float also accepts the strings “nan” and “inf” with an optional prefix “+” or “-” for Not a Number (NaN) and positive or negative infinity.
>>> a, b = float('nan'), float('-inf') >>> print a, b nan -inf
Some more basic mathematical functions along with the number types.
>>> pow(2,3) # 2^3 8 >>> 2**3 # 2^3 8 >>> x = 1.234 >>> import math # math module is needed >>> math.floor(x) 1.0 >>> math.ceil(x) 2.0
Unfortunately it is a little bit complicated to get the fractional part of a floating point number. >>> math.modf(x,1) # modulo function with rest (0.23399999999999999, 1.0) # results in a tuple >>> fx, fi = math.modf(x) # get both parts separately >>> fx # fractional part alone 0.23399999999999999
http://docs.python.org/library/stdtypes.html#typesnumeric
Python for Lectures
- 18 -
6.3 Strings
In order to execute some examples again start IDLE (see chap. 2.2).
IDLE 2.6.6 >>> 'text' 'text' >>> "this is a string" 'this is a string' >>> >>>
Single quotes (‘) and double quotes (“) define strings the same way.
The one sort can contain the other without being interpreted or changed.
>>> "Python's" "Python's" >>> 'This is in "double quotes"' 'This is in "double quotes"'
A raw string (r’text’) is a special string, where backslash commands are not interpreted and executed. It is e.g. useful to contain DOS-like path definitons (like r’c:\dir\file’).
>>> print r'c:\dir\file' c:\dir\file >>> print 'text with \n command sequence' text with command sequence
In contrast, in a string defined like 'text with \n command sequence' contains the backslash character and is interpreted as new line command.
For more information see e.g.: http://docs.python.org/tutorial/introduction.html#strings
6.4 Truth Value, Boolean Operations
The type object for Boolean type is named bool; the constructor for it takes any Python value and converts it to True or False.
>>> bool(1) True >>> bool(0) False >>> bool([]) False >>> bool( (1,) ) True
Any object can be tested for truth value, for use in an if or while condition or as operand of the Boolean operations below. The following values are considered false:
• None
• False
• zero of any numeric type, for example, 0, 0L, 0.0, 0j.
• any empty sequence, for example, '', (), [].
• any empty mapping, for example, {}.
Python for Lectures
- 19 -
• instances of user-defined classes, if the class defines a __nonzero__() or __len__() method, when that method returns the integer zero or bool value False. [1]
All other values are considered true — so objects of many types are always true.
Operations and built-in functions that have a Boolean result always return 0 or False for false and 1 or True for true, unless otherwise stated. (Important exception: the Boolean operations or and and always return one of their operands.)
Use any Boolean expression in an if or while statement:
if bool(1): print "yes,it is!" b = bool(0) print b type(b)
These are the Boolean operations , ordered by ascending priority:
Operation Result
x or y if x is false, then y, else x
x and y if x is false, then x, else y
not x if x is false, then True, else False
Comparison operations are supported by all objects. They all have the same priority (which is higher than that of the Boolean operations). Comparisons can be chained arbitrarily; for example, x < y <= z is equivalent to x < y and y <= z, except that y is evaluated only once (but in both cases z is not evaluated at all when x < y is found to be false).
This table summarizes the comparison operations:
Operation Meaning
< strictly less than
<= less than or equal
> strictly greater than
>= greater than or equal
== equal
!= not equal
is object identity
is not negated object identity
For more details see: http://docs.python.org/library/stdtypes.html#typesnumeric http://docs.python.org/release/2.5.2/lib/boolean.html
6.5 Declaring variables
Python has local and global variables like most other languages, but it has no explicit variable declarations. Variables spring into existence by being assigned a value, and they are automatically destroyed when they go out of scope.
>>> x # gives an error as it was not defined yet
Python for Lectures
- 20 -
Traceback (most recent call last): File "<pyshell#51>", line 1, in <module> x NameError: name 'x' is not defined >>> >>> x = 1 # the variable x is defined on the fly, by it’s first use >>> x 1
>>> price_of_fine_china_tea_no_1 = 2.50 # this variable name is allowed
You will thank Python for this simplicity one day.
Programmers generally choose names for their variables that are meaningful – they document what the variable is used for.
Variable names can be arbitrarily long. They can contain both letters and numbers, but they have to begin with a letter. Although it is legal to use uppercase letters, by convention we don’t. If you do, remember that case matters. Bruce and bruce are different variables.
The underscore character ( _) can appear in a name. It is often used in names with multiple words, such as my_name or price_of_fine_china_tea.
If you give a variable an illegal name, you get a syntax error:
>>> 76trombones = "big parade" SyntaxError: invalid syntax >>> more$ = 1000000 SyntaxError: invalid syntax >>> class = "Computer Science 101" SyntaxError: invalid syntax
76trombones is illegal because it does not begin with a letter. more$ is illegal because it contains an illegal character, the dollar sign. But what’s wrong with class?
It turns out that class is one of the Python keywords. Keywords define the language’s rules and structure, and they cannot be used as variable names.
For more see: http://diveintopython.org/native_data_types/declaring_variables.html chap 2.3 in http://openbookproject.net//thinkCSpy/ch02.html
7 Datatypes
7.1 Lists
A list is something like an array or a set of data stored with one variable name. In order to execute examples we work again with IDLE (see chap. 2.2).
IDLE 2.6.6 >>> a = [1, 2, 3, 4] >>> a [1, 2, 3, 4] >>> a[0] 1 >>> a[1] 2 >>>
Python for Lectures
- 21 -
“a” is defined here as a list with the four elements “1, 2, 3, 4”. The first element is retrieved by “a[0]”.
Note that Python starts always counting with number “0” for the first element!
This might be somewhat unusual and inconvenient for physicists, which are more used to start counting of natural objects with “1”. On the other hand, computer scientists work with bytes filled with all combinations from “000” to “111” in the binary system. Translated into the decimal system this corresponds to the numbers from 0 to 7. Thus for binary oriented work it is more obvious to start counting at “0”.
With the function “type” you can test the type of any variable.
>>> type(a) <type 'list'>
But a list can handle any object (and that’s everything in Python), not just numbers. Here we create a list of strings:
>>> b = ["a", "word", "b", "example"] >>> b[3] 'example' >>> b[1:3] ['word', 'b'] >>> b[1:4] ['word', 'b', 'example']
With “1:3” all elements from the first index, here number 1 (i.e. the second), up to, but not including the last index (up to number 2, which is the third element, because the counting starts at zero).
>>> b[-1] 'example'
If the index of a list is negative (i.e. “b[-1]”) counting starts from the end with “-1” corresponding to the last one.
Here somewhat more on Python counting. The “range” function creates a list with very much the same notation as the “0:3” retrieving of elements from a list.
>>> range(3) [0, 1, 2]
It also starts counting at “0” and ends one before the index “3” given as argument. Equivalently, the function can be simply seen as producing a list with 3 elements starting at 0.
For many more details see: http://diveintopython.org/native_data_types/lists.html http://docs.python.org/tutorial/introduction.html#lists
There are three built-in functions that are very useful when used with lists: filter(), map(), and reduce(), see: http://docs.python.org/tutorial/datastructures.html#functional-programming-tools
7.2 Tuples
A tuple is an immutable list. It cannot be changed in any way once it is created. Retrieving elements from a tuple is much the same way as it is with lists.
>>> b = ("a", "word", "b", "example") >>> b ('a', 'word', 'b', 'example')
Python for Lectures
- 22 -
>>> b[1] 'word' >>> >>> type(b) <type 'tuple'>
Tuples are faster than lists. Tuples can be converted into lists and vice-versa.
For more details see: http://diveintopython.org/native_data_types/tuples.html
7.3 Dictionaries
Unlike lists, which are indexed by a range of numbers, dictionaries are indexed by keys, which can be any immutable type; strings and numbers can always be keys.
>>> tel = {'Alex': 9600, 'Eugen':9605} >>> tel {'Eugen': 9605, 'Alex': 9600} >>> >>> tel['Alex'] 9600 >>> >>> tel['Tom'] = 9604 >>> tel {'Eugen': 9605, 'Alex': 9600, 'Tom': 9604} >>>
It is best to think of a dictionary as an unordered set of key: value pairs, with the requirement that the keys are unique (within one dictionary). A pair of braces creates an empty dictionary: {}. Placing a comma-separated list of key:value pairs within the braces adds initial key:value pairs to the dictionary; this is also the way dictionaries are written on output.
>>> tel.keys() ['Eugen', 'Alex', 'Tom'] >>> tel.values() [9605, 9600, 9604] >>> 'Tom' in tel True >>> del tel['Tom'] >>> 'Tom' in tel False >>>
The keys() method of a dictionary object (i.e. “tel.keys()” in the example above) returns a list of all the keys used in the dictionary, in arbitrary order (if you want it sorted, just apply the sorted() function to it). To check whether a single key is in the dictionary, use the in keyword. It is also possible to delete a key:value pair with del. If you store using a key that is already in use, the old value associated with that key is forgotten.
For many more details see: http://docs.python.org/tutorial/datastructures.html#dictionaries http://diveintopython.org/native_data_types/index.html#odbchelper.dict
Python for Lectures
- 23 -
8 Functions
Functions are very useful in order to execute program parts from different positions. The keyword def introduces a function definition.
8.1 One return value
In the next example, a circle is plotted in form of a parametric curve with parameter t being interpreted as time of a particle moving along. The velocity vector components are calculated by the derivatives dx/dt and dy/dt and is plotted along with the curve. In order to calculate the curve and the derivatives, a function containing the definition of the curve is called several times.
# prog_12_Curve_05.py # V05 parametric curve x(t), y(t); # t interpreted as time of particle along curve # v(t) derivatives dx/dx, dy/dt # save image in svg-file (scalable vector graphic) import matplotlib from pylab import * nmax = 4*8+1 # number of points along curve dt = 0.001 # interval size for derivative calculation om = 2*pi # angular velocity $\omega$ v_scale = 1/(2*pi) # scaling factor for velocity in x,y-space def fx(t,om): # define a function return cos(om*t) def fy(t,om): # define a function return sin(om*t) t = linspace(0,1,nmax); # time array (type is numpy.ndarray) x = fx(t,om) # function calls y = fy(t,om) x1 = fx(t-dt/2,om) # function calls x2 = fx(t+dt/2,om) dxodt = (x2-x1)/dt # calculate derivative y1 = fy(t-dt/2,om) # function calls y2 = fy(t+dt/2,om) dyodt = (y2-y1)/dt # calculate derivative figure(1, figsize=(8.0, 8.0)) # figure with size defined in inch plot(x, y, 'bo-', markersize=6, linewidth=1.5) # plot curve with markers for n in range(0,len(t),4): # plot V(t) for every 4th point plot([x[n],x[n]+v_scale*dxodt[n]], \ [y[n],y[n]+v_scale*dyodt[n]],'r-', linewidth=1.5) # plot statement is continued on 2nd line above axis('on'); grid('on') title('Parametric curve: circle with velocity vectors') xlabel('x'); ylabel('y')
Python for Lectures
- 24 -
savefig('prog_14_Curve_05.png', dpi=100) savefig('prog_14_Curve_05.svg', dpi=100) # scalable vector graphic show() # show plot
Execute the program from the SciTe editor or call the saved file from within the IPython shell or from IDLE.
Fig.8.1: Part of resulting figure of “prog_12_Curve_05.py”. The red lines represent velocity vectors.
8.2 No return value
In the function definition of the example above, a value (i.e. the function value) was returned to the calling position. Another possibility is that the function performs some task like printing, without returning anything. See the following example for printing the Fibonacci series.
# prog_13_function_Fibonacci.py def fib(n): # write Fibonacci series up to n """Print a Fibonacci series up to n.""" a, b = 0, 1 while a < n: print a, a, b = b, a+b # # Now call the function we just defined: fib(200)
It will print the result
0 1 1 2 3 5 8 13 21 34 55 89 144
For more info see : http://docs.python.org/tutorial/controlflow.html#defining-functions
8.3 Multiple return values
Most often when you have to return multiple arguments you will probably just use something simple like this example. Here we just take advantage of the fact that Python includes built in Tuples, Lists, and Dictionaries, and return one of these objects to encapsulate the multiple values. ( http://www.penzilla.net/tutorials/python/functions/ )
# multiple-returns.py a, b, c = 0, 0, 0 # multiple assignment for shortness def getabc(): a = "Hello" b = "World" c = "!" return a,b,c # defines a tuple on the fly def gettuple(): a,b,c = 1,2,3 # Notice the similarities between this and getabc? return (a,b,c) # defines a tuple by will
Python for Lectures
- 25 -
def getlist(): a,b,c = (3,4),(4,5),(5,6) return [a,b,c] # defines a list
These all work, as amazing as it seems.
So multiple assignment is actually quite easy.
a,b,c = getabc() d,e,f = gettuple() g,h,i = getlist()
8.4 Default argument values
The most useful form is to specify a default value for one or more arguments. This creates a function that can be called with fewer arguments than it is defined to allow. For example:
def ask_ok(prompt, retries=4, complaint='Yes or no, please!'): while True: ok = raw_input(prompt) if ok in ('y', 'ye', 'yes'): return True if ok in ('n', 'no', 'nop', 'nope'): return False retries = retries - 1 if retries < 0: raise IOError('refusenik user') print complaint
This function can be called in several ways: * giving only the mandatory argument: ask_ok('Do you really want to quit?') * giving one of the optional arguments: ask_ok('OK to overwrite the file?', 2) * or even giving all arguments: ask_ok('OK to overwrite the file?', 2, 'Come on, only yes or no!')
This example also introduces the in keyword. This tests whether or not a sequence contains a certain value.
The default values are evaluated at the point of function definition in the defining scope, so that
i = 5 def f(arg=i): print arg i = 6 f()
will print
5
Important warning: The default value is evaluated only once at the function definition. This makes a difference when the default is a mutable object such as a list, dictionary, or instances of most classes. For example, the following function accumulates the arguments passed to it on subsequent calls:
def f(a, L=[]): L.append(a) return L
Python for Lectures
- 26 -
print f(1) print f(2) print f(3)
This will print
[1] [1, 2] [1, 2, 3]
If you don’t want the default to be shared between subsequent calls, you can write the function like this instead:
def f(a, L=None): if L is None: L = [] L.append(a) return L
More on defining functions: http://docs.python.org/tutorial/controlflow.html#more-on-defining-functions
8.5 Lambda expressions
Lambda expressions are also called anonymous functions. Python supports an interesting syntax that lets you define one-line mini-functions on the fly. Borrowed from Lisp, these so-called lambda functions can be used anywhere a function is required.
# prog_14_lambda_expression.py def f(x): return x*2 print f(3) y = lambda x: x*2 print y(3)
The lambda function accomplishes the same thing as the normal function above it. Note the abbreviated syntax in the lambda expression: there are no parentheses around the argument list, and the return keyword is missing (it is implied, since the entire function can only be one expression). Also, the function has no name, but it can be called through the variable it is assigned to. (http://diveintopython.org/power_of_introspection/lambda_functions.html )
8.6 Built-in functions
There are many built-in-functions available in Python (http://docs.python.org/library/functions.html )
There of them are very useful when used with lists: filter(), map(), and reduce().
filter(function, sequence) returns a sequence consisting of those items from the sequence for which function(item) is true. If sequence is a string or tuple, the result will be of the same type; otherwise, it is always a list. For example, to compute primes up to 25:
>>> def f(x): return x % 2 != 0 and x % 3 != 0 ... >>> filter(f, range(2, 25)) [5, 7, 11, 13, 17, 19, 23]
Python for Lectures
- 27 -
map(function, sequence) calls function(item) for each of the sequence’s items and returns a list of the return values. For example, to compute some cubes:
>>> def cube(x): return x*x*x ... >>> map(cube, range(1, 11)) [1, 8, 27, 64, 125, 216, 343, 512, 729, 1000]
More than one sequence may be passed; the function must then have as many arguments as there are sequences and is called with the corresponding item from each sequence (or None if some sequence is shorter than another). For example:
>>> seq = range(8) >>> def add(x, y): return x+y ... >>> map(add, seq, seq) [0, 2, 4, 6, 8, 10, 12, 14]
reduce(function, sequence) returns a single value constructed by calling the binary function function on the first two items of the sequence, then on the result and the next item, and so on. For example, to compute the sum of the numbers 1 through 10:
>>> def add(x,y): return x+y ... >>> reduce(add, range(1, 11)) 55
If there’s only one item in the sequence, its value is returned; if the sequence is empty, an exception is raised.
See also: http://docs.python.org/tutorial/datastructures.html#functional-programming-tools
9 Important built-in modules
Python owns many built-in modules. In order to use them inside a user program, they have to be imported first.
The sys module allows access to interpreter specific and PC platform specific informations.
The os module is an interface to the computer operating system and enables the call of shell commandos and more.
The string module contains a number of functions to process standard Python strings.
The time module provides functions to work with dates and times.
The re module provides support to recognize character patterns inside strings and to extract substrings.
The math module implements a number of mathematical operations for floating point numbers.
The cmath module provides similar functionality for complex numbers.
The fileinput module makes it easy to write different kinds of text filters.
The random module provides a number of different random number generators.
Multimedia Modules: Python comes with a small set of modules for dealing with image files and audio files.
Python for Lectures
- 28 -
For more extensive support of multimedia , see Python Imaging Library (PIL) and the Snack Sound Toolkit, among others.
See http://docs.python.org/tutorial/stdlib.html and http://effbot.org/librarybook/
Because Python comes with a large amount of modules included, it is often stated as “Batteries included”.
9.1 The sys module
One module that provides insightful information about Python itself is the sys module. You make use of a module by importing the module and referencing its contents (such as variables, functions, and classes) using dot (.) notation.
The sys module contains a variety of variables and functions that reveal interesting details about the current Python interpreter. Let's take a look at some of them. The first thing we'll do is import the sys module and ask for sys.platform.
Again, we're going to run Python interactively and enter commands at the Python command prompt. The first thing we'll do is import the sys module. Then we'll enter the sys.executable variable, which contains the path to the Python interpreter:
If one runs Python interactively and enters commands at the Python command prompt, this looks like
>>> import sys >>> sys.executable '/usr/local/bin/python'
We can do the same if we write a small program in the editor and then execute the file.
import sys print sys.platform
This prints
win32
(or any other computer platform, we are working on) into the output window.
The following program gives additional information on the sys module and shows access to different data types.
# prog_15_sys_module.py # [email protected] # see also http://www.ibm.com/developerworks/library/l-pyint.html import sys print "computer platform: ", sys.platform # computer platform: win32, linux, ... print "python interpreter: ", sys.executable # python interpreter used print "phython version:", sys.version_info # python version # print sys.argv # print sys.path print "" # empty line print "In order to activate the output window,\ click with mouse into this window here" # this statement extends over two lines raw_input("and then press return to continue!\n") print sys.modules # this gives a long line containing actually a python dictionary print "the type of sys.modules is: ", type(sys.modules)
Python for Lectures
- 29 -
print "" raw_input("Press return to continue!") print "'dictionary.items()' converts the content into a list" L = sys.modules.items() print "the list L contains %g items" % len(L) print "printing the first 3 items (each being a tuple itself):" for nn in range(3): print L[nn] ## print whole list #print L ## print all entries of dictionary sys.modules #for entry1, entry2 in sys.modules.items(): #print " %s: %s " % (entry1, entry2)
In order to get the name of the currently executed (interpreted) python file:
import os,sys py_file = sys.argv[0] # list of system arguments for python interpreter
9.2 The os module
For a systematic overview see: http://docs.python.org/library/os.html
A very good introduction in form of examples is given in http://effbot.org/librarybook/os.htm. We look at some of these (somewhat modiefied) examples here.
Using the os module to list the files in a directory with os.listdir(path):
# prog_16_os_1.py import os for file in os.listdir(""): # list current working directory if path string is empty print file
results in
prog_01.py prog_02.py ...
On Windows one can e.g. give the path string like "C:\\temp" to get the directory listing of “temp” on “C:”. Note that you have to use “\\” in a string in order to produce a single “\” in the command output. A single backslash in the string would here be interpreted as the tabulator command “\t”.
for file in os.listdir("C:\\temp"): # note the “\\” under Windows print file
Using the os module to create and remove a directory and to change the working directory:
import os # where are we? cwd = os.getcwd() print "1", cwd os.mkdir("test") # create subdir "test"
Python for Lectures
- 30 -
# go down os.chdir("test") print "2", os.getcwd() # go back up os.chdir(os.pardir) print "3", os.getcwd() os.rmdir("test") # removes only empty directories # will fail, as subdir has just been removed os.chdir("test") print "2", os.getcwd()
Using the os module to create and remove multiple directory levels and create a text file meanwhile:
# prog_16_os_2b.py # http://effbot.org/librarybook/os.htm # Using the os module to create and remove multiple directory levels # File: os-example-6.py import os # create several directory levels at once os.makedirs("test/multiple/levels") fp = open("test/multiple/levels/file.txt", "w") # create text file fp.write("this text is in file") # write into text file fp.close() # close file access # list files in directory for file in os.listdir("test/multiple/levels"): print file # remove the file os.remove("test/multiple/levels/file.txt") # and all empty directories above it os.removedirs("test/multiple/levels")
Note that removedirs removes all empty directories along the given path, starting with the last directory in the given path name. In contrast, the mkdir and rmdir functions can only handle a single directory level. Using the os module to get information about a file:
# prog_16_os_3.py # File: os-example-1.py import os import time file = "prog_01.py" def dump(st): mode, ino, dev, nlink, uid, gid, size, atime, mtime, ctime = st print "- size:", size, "bytes" print "- owner:", uid, gid print "- created:", time.ctime(ctime) print "- last accessed:", time.ctime(atime)
Python for Lectures
- 31 -
print "- last modified:", time.ctime(mtime) print "- mode:", oct(mode) print "- inode/dev:", ino, dev # get stats for a filename stat_info = os.stat(file) print "\nstat_info", file #print stat_info dump(stat_info) print
Using the os module to run an operating system command:
# prog_16_os_4.py # File: os-example-8.py import os if os.name == "nt": command = "dir" else: command = "ls -l" os.system(command)
Further examples on http://effbot.org/librarybook/os.htm : Using the os module to change a file’s privileges and timestamps Using the os module to run an operating system command Using the os module to start a new process Using the os module to run another program (Unix) Using the os module to run another program (Windows) Using the os module to run another program in the background (Windows) Using the os module to exit the current process
10 Pitfalls
10.1 The object pitfall
Remember? Everything in Python is an object! This may cause troubles in some cases.
What is the problem with:
>>> a = [1, 2, 3] >>> b = a >>> a.append(4) # the number 4 is appended to the list >>> print b # b has changed to [1, 2, 3, 4] as well! [1, 2, 3, 4]
At first something on the objects and their identity:
>>> a = 3 >>> id(3) # identity number of object 10017632 >>> id(a)
Python for Lectures
- 32 -
10017632
Here the number “3” is assigned to variable “a”. So the number “3” is generated as an object with a certain (more or less arbitrary?) integer identity number. The variable “a” gets the same identity number as it is equal to number “3”.
>>> b = a >>> id(b) 10017632 >>> print b 3
With the assignment “b = a”, variable “b” gets the same id-number as “3” and “a”. And if you print b, you get “3” as an output.
>>> a = 2 >>> id(2) 10017644 >>> id(a) 10017644 >>> id(b) 10017632
Now we changed the assignment of “a” to the object “2”, and indeed both have an new, identical id-number. The id-number of “b” from above has not been changed, as it points to the object “3”, which is an immutable object and still exists in the workspace. Everything works as expected.
But this is not always the case. Lists are objects that can be changed at place (mutable objects) and therefore unwanted effects may occur.
See the example from above:
>>> a = [1, 2] >>> b = a >>> id(a) 18496008 >>> a.append(3) # the number 3 is appended to the list >>> id(a) 18496008 # the object id is not changed! >>> print b # b “has changed” to [1, 2, 3] as well! [1, 2, 3]
Here “a” “b” and the list “[1, 2]” all have the same object id after the first two lines. Then by a.append, the list is changed under the same object identity. So everything which refers to that object now refers to the changed list – so does the variable “b”.
For more explanation on that see “Python Objects” at http://effbot.org/zone/python-objects.htm and “Sharing mutable objects” at http://www.python.org/doc/essays/ppt/hp-training/sld034.htm.
To overcome the problem above, instead of “b = a” it is necessary to create a separate object for “b” with the same content. This can be achieved in different ways:
>>> a = [1, 2] >>> id(a) 18540104 >>> b = a[:] # slicing: alle elements of “a” are taken and assigned to “b” >>> id(b) # “b” is now a new, independent object 13231288 >>> c = list(a) # creates a new list out of “a” >>> id(c) # “c” is a new object 18522440 >>> print c [1, 2] >>>
Python for Lectures
- 33 -
>>> import copy # the copy module is built to copy lists and objects, etc. >>> # it possesses different copy methods >>> d = copy.copy(a) >>> d [1, 2] >>> id(d) # “d” has a new id-number, i.e. is assigned to a new list object 18522160 >>> id(a) # id of “a” is still the same as at beginning! 18540104
For more details on object copies see: http://stackoverflow.com/questions/2612802/how-to-clone-a-list-in-python http://www.velocityreviews.com/forums/t684012-python-copy-method-alters-type.html http://docs.python.org/library/copy.html http://openbook.galileocomputing.de/python/python_kapitel_17_006.htm#mjc96b1149114c87b8cb9ce33af7a7f2d0 (german language)
A tuble is immutable and thus copy does not create a new object:
# prog_17_copy_1.py from copy import * f = (1, 2) # a tuple is immutable g = copy(f) # does not create a new object! print id(f) # identity number of object print id(g) # shows same id-number as f print f is g # results in ‘True’
10.2 More pitfalls
For „10 Python pitfalls” see: http://zephyrfalcon.org/labs/python_pitfalls.html
11 Introspection
Guide to Python introspection: http://www.ibm.com/developerworks/library/l-pyint.html
(Introspections means in German language: Selbstbeobachtung, Innenschau, … )
to be continued ...
12 Appendix
Details, examples and applications are collected here.
Python for Lectures
- 34 -
12.1 Conversion of a Matlab file into a Python file
Here are some important conversion rules which I recognized (and worked):
replace by comment comment % # commenting
.* * element by element
.^ ** power operator
; no line endings
length() len() length of array
figure(1) figure(1,(8,6),90) num=1, figsize=(8,6), dpi=90
more possibilities
hold on hold(True) for figures
... show() to display figures
in ind „in“ is a keyword in
x(in) x[ind] use []-brackets
a = b a = b[:] new independ. obj. pitfall!
for in = 1:length(x2)
for ind in range(len(x2)):
See also „Python Lists and Arrays“ at http://www.brpreiss.com/books/opus7/html/page82.html
See also: http://www.scipy.org/Cookbook/Matplotlib/Animations http://www.scipy.org/Cookbook/Matplotlib, http://www.scipy.org/Cookbook ,
and: http://bytes.com/groups/python/44332-there-better-interactive-plotter-then-pylab
12.2 Figures with matplotlib
http://seehuhn.de/blog/59
2009-02-03
I like to use the matplotlib Python library to create figures for use in my articles. Unfortunately, the matplotlib default settings seem to be optimised for screen-viewing, so creating pictures for inclusion in a scientific paper needs some fiddling. Here is an example script which sets up things for such a figure.
#! /usr/bin/env python import matplotlib matplotlib.use("PS") from pylab import * rc("font", family="serif") rc("font", size=10) width = 4.5 height = 1.4
Python for Lectures
- 35 -
rc("figure.subplot", left=(22/72.27)/width) rc("figure.subplot", right=(width-10/72.27)/width) rc("figure.subplot", bottom=(14/72.27)/height) rc("figure.subplot", top=(height-7/72.27)/height) figure(figsize=(width, height)) x = load("x.dat") plot(x[:,0], x[:,1], "k-") savefig("figure.eps")
The script performs the following steps:
* The matplotlib.use statement tells matplotlib to use one of the non-interactive backends. Without this statement, the script would need a connections to an X-windows server to run.
* The two rc statement set the font family and size for the labels on the axes to match the font used in the article's text (matplotlib uses a 12pt sans-serif font by default).
* The rc("figure.subplot", ...) settings and the figsize argument to the figure call set the image geometry. Since the text on journal pages typically is about 5in wide, a total figure width of 4.5in is often a good value. The margins set by the rc statements have to be given as fractions of the image width/height and must be big enough to contain the axis labels. A bit of experimentation helps to find good values; here we use a 14pt margin at the bottom (10pt font height plus some space), 22pt left margin (depends on the number of digits in the y-axis labels), 7pt for the top margin (space for half of the top-most y-axis label, and then some more) and 10pt to the right (enough space for half the right-most x-axis label).
* The color code "k" in the plot command sets the plot color to black. Matplotlib default is blue, but some journals charge you for use of colour in your articles (and also it is nice if you can print things on a b/w printer).
* The savefig command creates an "encapsulated PostScript file". This format is good for inclusion in TeX documents. The file can be converted to PDF using the epstopdf command as needed.
This is an excerpt from Jochen's blog.
12.3 Nonblocking Plots with Matplotlib
Is there a way to detach matplotlib plots so that t he computation can continue?
http://stackoverflow.com/questions/458209/is-there-a-way-to-detach-matplotlib-plots-so-that-the-computation-can-continue
After these instructions in the Python interpreter one gets a window with a plot:
from matplotlib.pyplot import * plot([1,2,3]) show() # other code
Unfortunately, I don't know how to continue to interactively explore the figure created by show() while the program does further calculations.
Is it possible at all? Sometimes calculations are long and it would help if they would proceed during examination of intermediate results.
Use matplotlib's calls that won't block:
Using draw():
Python for Lectures
- 36 -
from matplotlib import plot, draw, show plot([1,2,3]) draw() print 'continue computation' # it’s not blocked – do anything here instead! show() # at the end call show to ensure window won't close.
Using interactive mode:
from matplotlib import plot, ion, show ion() # enables interactive mode plot([1,2,3]) # result shows immediatelly (implicit draw()) print 'continue computation' # it’s not blocked – do anything here instead! show() # at the end call show to ensure window won't close.
12.4 Matplotlib's OO Class Library
For those people coming to Matplotlib without any prior experience of MatLab and who are familiar with the basic concepts of programming API's and classes, learning to use Matplotlib via the class library is an excellent choice. The library is well put together and works very intuitively once a few fundamentals are grasped.
The advice from John Hunter, the original developer of the library is 'don't be afraid to open up matplotlib/pylab.py to see how the pylab interface forwards its calls to the OO layer.' That in combination with the user's guide, the examples/embedding demos, and the mailing lists, which are regularly read by many developers are an excellent way to learn the class library.
Following is a brief introduction to using the class library, developed as I came to grips with how to produce my first graphs.
For getting started with Matplotlib's OO Class Library see: http://matplotlib.sourceforge.net/leftwich_tut.txt
12.5 File and Directory handling/manipulation
Check if an entry is a file or directory in Python: http://lookherefirst.wordpress.com/2007/12/03/check-if-an-entry-is-a-file-or-directory-in-python/
To check whether an entry is a file or a directory in python, use the os.path.isdir() method. To read some file attributes, use the os.stat() method. The following code will read all entries in the current directory and then print out whether the entry is a file or a directory together with the time the file/dir was created.
import os, time dirList = os.listdir("./") for d in dirList: if os.path.isdir(d) == True: stat = os.stat(d) created = os.stat(d).st_mtime asciiTime = time.asctime( time.gmtime( created ) ) print d, "is a dir (created", asciiTime, ")"
Python for Lectures
- 37 -
else: stat = os.stat(d) created = os.stat(d).st_mtime asciiTime = time.asctime( time.gmtime( created ) ) print d, "is a file (created", asciiTime, ")"
A simple non-recursive directory walker: http://code.activestate.com/recipes/435875/
This is also usefull for incremental directory walking (where you want to scan remaining directories in subsequent calls).
import os, os.path startDir = "/" directories = [startDir] while len(directories)>0: directory = directories.pop() for name in os.listdir(directory): fullpath = os.path.join(directory,name) if os.path.isfile(fullpath): print fullpath # That's a file. Do something with it. elif os.path.isdir(fullpath): directories.append(fullpath) # It's a directory, store it.
12.6 Measuring raw speed
Here's one way (http://www.python.org/doc/essays/ppt/hp-training/sld041.htm ):
import time def timing(func, arg, ncalls=100): r = range(ncalls) t0 = time.clock() for i in r: func(arg) t1 = time.clock() dt = t1-t0 print "%s: %.3f ms/call (%.3f seconds / %d calls)" \ % ( func.__name__, 1000*dt/ncalls, dt, ncalls)
Using the profile module: http://www.python.org/doc/essays/ppt/hp-training/sld040.htm
12.7 Video processing
See: http://www.google.at/#sclient=psy&hl=de&q=python+video+module
http://pymedia.org/tut/:
PyMedia is a Python library for accessing and manipulating media files. It makes audio and video playback/creation a snap for even a newcomer to programming. This tutorial aims to walk you through installing and using the PyMedia library. For the sake of simplicity most of the examples have been kept concise and straightforward. In a 'real world' application, you would need to take care of error handling, input validation and so on.
Python for Lectures
- 38 -
http://pymedia.org/tut/src/dump_video.py.html:
#! /bin/env python import sys, os import pymedia.muxer as muxer import pymedia.video.vcodec as vcodec import pygame def dumpVideo( inFile, outFilePattern, fmt ): dm= muxer.Demuxer( inFile.split( '.' )[ -1 ] ) i= 1 f= open( inFile, 'rb' ) s= f.read( 400000 ) r= dm.parse( s ) v= filter( lambda x: x[ 'type' ]== muxer.CODEC_TYPE_VIDEO, dm.streams ) if len( v )== 0: raise 'There is no video stream in a file %s' % inFile v_id= v[ 0 ][ 'index' ] print 'Assume video stream at %d index: ' % v_id c= vcodec.Decoder( dm.streams[ v_id ] ) while len( s )> 0: for fr in r: if fr[ 0 ]== v_id: d= c.decode( fr[ 1 ] ) # Save file as RGB BMP if d: dd= d.convert( fmt ) img= pygame.image.fromstring( dd.data, dd.size, "RGB" ) pygame.image.save( img, outFilePattern % i ) i+= 1 s= f.read( 400000 ) r= dm.parse( s ) print 'Saved %d frames' % i # ---------------------------------------------------------------------------------- # Dump the whole video file into the regular BMP images in the directory and file name specified # http://pymedia.org/ if __name__ == "__main__": if len( sys.argv )!= 4: print 'Usage: dump_video <file_name> <image_pattern> <format_number>\n<format_number> can be: RGB= 2'+\ '\n<image_patter> should include %d in the name. ex. test_%d.bmp.'+ \ '\nThe resulting image will be in a bmp format' else: pygame.init() dumpVideo( sys.argv[ 1 ], sys.argv[ 2 ], int( sys.argv[ 3 ] ) ) pygame.quit()
http://stackoverflow.com/questions/1480431/most-used-python-module-for-video-processing: I used this script to convert a movie to a numpy array + binary store:
"""Takes a MPEG movie and produces a numpy record file with a numpy array.""" import os filename = 'walking' if not(os.path.isfile(filename + '.npy')): # do nothing if files exists N_frame = 42 # number of frames we want to store os.system('ffmpeg -i WALK.MOV.qt -f image2 foo-%03d.png') # convert them to numpy from numpy import zeros, save, mean from pylab import imread
Python for Lectures
- 39 -
n_x, n_y, n_rgb = imread('foo-001.png').shape mov = zeros((n_y, n_x, N_frame)) for i_frame in range(N_frame): name = 'foo-%03d.png' % (i_frame +1) mov[:n_y,:n_x,i_frame] = flipud(mean(imread(name), axis=2)).T os.system('rm -f foo-*.png') save(filename + '.npy', mov)
note that depending on your conventions you may not want to flip the image. you may then load it using :
load('walking.npy')
13 Other stuff
This is a deposit of infos, materials, interesting details that might be worth to have a look at or to incorporate it into the main part of this script.
13.1 Unordered Stuff
Python tips by Guide von Rossum: http://www.python.org/doc/essays/ppt/hp-training/sld019.htm
Common Tasks in Python: http://oreilly.com/catalog/lpython/chapter/ch09.html
Popular Python recipes: http://code.activestate.com/recipes/langs/python/
The Standard Python Library: http://effbot.org/librarybook/
Date and Time Representation in Python: http://seehuhn.de/pages/pdate
Asynchronous plotting with threads: http://esclab.tw/wiki/index.php/Matplotlib
Python Information and Examples: http://www.secnetix.de/olli/Python/
An Introduction to Python: http://www.penzilla.net/tutorials/python/intro/index.shtml
Integrated Development Environments: http://www.penzilla.net/tutorials/python/ide/
NumPy for MATLAB users: http://mathesaurus.sourceforge.net/matlab-numpy.html
Minimal Python for Scientific Computing: http://www.physics.ucf.edu/~mdj/MinimalPython.html
Advantages of Python:
Here is some motivation, why to go on and learn Python!
Why python? http://www.linuxjournal.com/article/3882
Python as a First Language: http://mcsp.wartburg.edu/zelle/python/python-first.html
Java vs. Python: http://hathawaymix.org/Weblog/2004-06-16
Number arrays: I thought that only pylab can work with arrays, but also scimath can do it:
# prog_08_module_4.py
Python for Lectures
- 40 -
import pylab as p # include function e.g. as p.sin(0) import numpy.lib.scimath as s x = p.arange(0,3,1) # creates an array print x y1 = p.sqrt(x) print y1 y2 = s.sqrt(x) print y2 x2 = p.arange(0,3,1) y21 = x2**2 print y21 x3 = [0., 1., 4.] # creates a list y31 = x3**2 print y31 #x = p.linspace(0,2*pi,50) # defines an array
http://www.brpreiss.com/books/opus7/html/page82.html: Python Lists and Arrays
http://assorted-experience.blogspot.com/2007/07/initializing-arrays-in-pylab-numpy.html: Initializing arrays in pylab (numpy): When you initialize an array say using zeros or ones, the array constructor remembers that the default type is of integer type. So if you plan on doing floating point operations always set the type explicitly like x = zeros(10,'f') or y = array([ ],'f')
http://assorted-experience.blogspot.com/
More:
What are scipy, numpy, matplotlib ?
Python is a general purpose programming language. It is interpreted and dynamically typed and is very suited for interactive work and quick prototyping, while being powerful enough to write large applications in.
Numpy is a language extension that defines the numerical array and matrix type and basic operations on them.
Scipy is another language extension that uses numpy to do advanced math, signal processing, optimization, statistics and much more.
Matplotlib is a language extension to facilitate plotting.
http://matplotlib.sourceforge.net/
http://www.scipy.org/PyLab
http://www.scipy.org/Getting_Started
Plotting with Pylab: http://www.daniweb.com/code/snippet216915.html
Nice, important:
Python for Lectures
- 41 -
http://compwiki.physics.utoronto.ca/Physics+with+Pylab
How to Think Like a Computer Scientist – Learning w ith Python:
http://www.rg16.asn-wien.ac.at/~python/how2think/english/index.htm
Measure execution time of small code snippets http://docs.python.org/library/timeit.html
14 Version changes
V04:
New enumeration of chapters!!!
In the first part of the new version of the script, important changes are highlighted in green color in order to help finding these parts having worked with the previous version.
Changes:
2.2 IDLE
3.2 Tutorials: Tutorial info added at end of chapter.
4.24.2 Plotting: Cut-out of images Hint on “NumPy for MATLAB users” added at end of chapter.
5.3 Hint on indentation with SciTe editor
6 Numbers, Strings, Boolean – completely new part
7 Datatypes – completely new part
8 Functions – completely new part
9 The sys module and introspection – not changed
V05:
Table of content is shifted in front of foreword – is not correct but more practical, if script is used online.
9 Important built-in modules: sys, os
10 Pitfalls
11 Introspection: not yet realized
12 Appendix: contains details, special topics
13 Other stuff: might be useful
14 This version info here
