Upload
ilian-iliev
View
213
Download
1
Embed Size (px)
DESCRIPTION
python course
Citation preview
Steven Bamford
An introduction to scientific programming with
http://stevenbamford.com/pythoncourse
• Part 1: Introduction • Why (and why not) to use a modern, high-level, scripting language • Why Python is awesome • Introduction to the language
• startup, syntax, constructs, functions, classes, getting help • Good programming practice versus 'thinking aloud’ • Python 2.x versus 3.x
• Part 2: Scientific programming in Python • Arrays – Numerical Python • Using arrays wisely • Plotting • Scientific Python
• optimisation, interpolation, statistics, filtering, integration, … • Other tools
• GNU Scientific Library, R, … • An example analysis program
• Part 3: Python for observers • Handling FITS files • PyRAF – scripting IRAF with Python
Part 1: Introduction
An introduction to scientific programming with
• Modern scripting languages: • Python, Perl, Ruby, IDL, … • High-level • Interactive interpreter
• Ease of use
• Speed of development
• Encourages scripting, rather than one-off analysis
• Permanent record
• Repeatability
• If you want fastest possible performance
• Highly parallel code
• Need low-level control
• Designed to be easy to learn and use – clear syntax
• Well documented
• Powerful, flexible, fully-featured programming language
• ‘Batteries included’
• Comprehensive scientific tools
• Fast
• Interpreter, introspection
• Runs everywhere
• Completely free
• You already have it
• Less stress
• Get more science done
• Widely used and growing popularity
• Throughout academia and industry • NASA, AstraZeneca, Google, Industrial Light & Magic, Philips,… • Web services, engineering, science, air traffic control, quantitative finance,
games, education, data management, …
• Python programmers in demand
• Easy introduction to general programming concepts
Why not?
• Existing code for your project in another language, but still…
http://python.org
>>> 2+2 4 >>> # This is a comment ... 2+2 4 >>> 2+2.0 # and a comment on the same line as code 4.0 >>> (50-5*6)/4 5 >>> width = 20 # assignment, no type declaration >>> height = 5*9 >>> width * height 900 >>> x = y = z = 0 # zero x, y and z >>> y 0 >>> n Traceback (most recent call last): File "<stdin>", line 1, in <module> NameError: name 'n' is not defined
2+2 # This is a comment 2+2 2+2.0 # and a comment on the same line as code (50-5*6)/4 width = 20 # assignment, no type declaration height = 5*9 width * height x = y = z = 0 # zero x, y and z y
• Can write in a text editor and copy and paste into interpreter
• Can save and execute from command line: $ python test.py
• Can save and use interactively in future sessions (import)
>>> 'spam and eggs' 'spam and eggs' >>> 'doesn\'t' "doesn't" >>> "doesn't" "doesn't" >>> '"Yes," he said.' '"Yes," he said.’ >>> hello = ‘Greetings!’ >>> hello ‘Greetings!’ >>> print(hello) Greetings! >>> print(hello + ‘ How do you do?’) Greetings! How do you do? >>> print(hello, ‘How do you do?’) Greetings! How do you do? >>> howdo = ’How do you do?’ >>> print(hello+’ ‘+howdo) Greetings! How do you do?
>>> 10 + 3 13 >>> 10 - 3 7 >>> 10 * 3 30 >>> 10 / 3 3 >>> 10 // 3 3 >>> 10 % 3 1 >>> 10**3 1000 >>> 10 + 3 * 5 # *,/ then +,- 25 >>> (10 + 3) * 5 65 >>> -1**2 # -(1**2) -1
>>> 10.0 + 3.0 13.0 >>> 10.0 - 3.0 7.0 >>> 10.0 * 3 30.0 >>> 10.0 / 3 3.3333333333333335 >>> 10.0 // 3 3.0 >>> 10.0 % 3.0 1.0 >>> 10.0**3 1000.0
>>> 4.2 + 3.14 7.3399999999999999 >>> 4.2 * 3.14 13.188000000000001
Augmented assignment: >>> a = 20 >>> a += 8 >>> a 28 >>> a /= 8.0 >>> a 3.5
Functions: >>> abs(-5.2) 5.2 >>> sqrt(25) 5.0
Comparisons: >>> 5 * 2 == 4 + 6 True >>> 0.12 * 2 == 0.1 + 0.14 False >>> a = 0.12 * 2; b = 0.1 + 0.14 >>> eps = 0.0001 >>> a - eps < b < a + eps True
Lists: >>> a = [1, 2, 4, 8, 16] # list of ints >>> c = [4, ‘candles’, 4.0, ‘handles’] # can mix types >>> c[1] ‘candles’ >>> c[2] = ‘knife’ >>> c[-1] # negative indices count from end ‘handles’ >>> c[1:3] # slicing [‘candles’, ‘knife’] >>> c[2:] # omitting defaults to start or end [‘knife’, ‘handles’] >>> c[0:4:2] # variable stride (could just write c[::2]) [4, ‘knife’] >>> a + c # concatenate [1, 2, 4, 8, 16, 4, ‘candles’, ‘knife’, ‘handles’] >>> len(a) 5
Tuples: >>> q = (1, 2, 4, 8, 16) # tuple of ints >>> r = (4, ‘candles’, 4.0, ‘handles’) # can mix types >>> s = (‘lonely’,) # singleton >>> t = () # empty >>> r[1] ‘candles’ >>> r[2] = ‘knife’ # cannot change tuples Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: 'tuple' object does not support item assignment
>>> u = 3, 2, 1 # parentheses not necessary
>>> v, w = ‘this’, ‘that >>> v ‘this’ >>> w ‘that’
Dictionaries: >>> a = {‘eyecolour’: ‘blue’, ‘height’: 152.0, 42: ‘the answer‘} >>> a[‘age’] = 28 >>> a {42: 'the answer', 'age': 28, 'eyecolour': 'blue', 'height': 152.0}
>>> del(a[‘height’]) >>> a {42: 'the answer', 'age': 28, 'eyecolour': 'blue'}
>>> b = {} >>> b[‘hello’] = ‘Hi!’
>>> a.keys() [42, 'age’, 'eyecolour’] >>> a.values() ['the answer’, 28, 'blue’]
>>> a = 4; b = 3 >>> if a > b: ... result = ‘bigger’ ... c = a - b ... >>> print(result, c) bigger 1
>>> a = 1; b = 3 >>> if a > b: ... result = ‘bigger’ ... elif a == b: ... result = ‘same’ ... else: # i.e. a < b ... result = ‘smaller’ ... >>> print(result) smaller
>>> if a < b: print ‘ok’ ok
Comparison operators:
== != > < >= <= is is not in not in
Boolean operators:
and or not
• Indentation is important! • be consistent • use four spaces • do not use tabs
>>> if ‘Steven’ in [‘Bob’, ‘Amy’, ‘Steven’, ‘Fred’]: ... print ‘Here!’ ... Here!
>>> if ‘Carol’ not in [‘Bob’, ‘Amy’, ‘Steven’, ‘Fred’]: ... print ‘Away!’ ... Away!
>>> test = a == b >>> if test: print ‘Equal’ ‘Equal’
>>> a = b = 0 >>> while a < 10: ... a += 3 ... print(a) ... 3 6 9 12
>>> while True: ... b += 3 ... if b >= 10: break ... print(b) 3 6 9
>>> for i in [2, 5, 3]: ... print(i**2) 4 25 9
>>> for j in range(5): print(j) 0 1 2 3 4
>>> range(3, 10, 2) [3,5,7,9]
>>> d = {‘this’: 2, ‘that’: 7} >>> for k, v in d.items(): ... print(‘%s is %i’%(k, v)) this is 2 that is 7
>>> def my_func(x, y=0.0, z=1.0): ... a = x + y ... b = a * z ... return b ... >>> my_func(1.0, 3.0, 2.0) 8.0 >>> my_func(1.0, 3.0) 4.0 >>> my_func(1.0, y=3.0) 4.0 >>> my_func(5.0) 5.0 >>> my_func(2.0, z=3.0) 6.0 >>> my_func(x=2.0, z=3.0) 6.0
>>> a = [2, 5, 3, 6, 5] >>> a.sort() >>> print(a) [2, 3, 5, 5, 6] >>> a.count(3) 1 >>> a.count(5) 2 >>> a.reverse() >>> print(a) [6, 5, 5, 3, 2]
>>> d = {‘black’: 100, ‘grey’: 50, ‘white’: 0} >>> d.values() [0, 50, 100]
>>> s = ‘-‘.join((‘2009’, ‘07’, ‘07’)) >>> print(s) 2009-07-07
• A neat way of creating lists (and arrays) without writing a loop
my_fav_num = [3, 17, 22, 46, 71, 8]
even_squared = [] for n in my_fav_num: if n%2 == 0: even_squared.append(n**2)
# in one line: even_better = [n**2 for n in my_fav_num if n%2 == 0]
# both produce [484, 2116, 64]
freshfruit = [' banana', ' loganberry ', 'passion fruit ']
stripped = [weapon.strip() for weapon in freshfruit]
print(stripped)
['banana', 'loganberry', 'passion fruit']
>>> class MyClass: ... def __init__(self, x, y=1.0) ... self.my_x = x ... self.my_y = y ... ... def product(): ... return x*y ... >>> m = MyClass(2, 4) >>> m.product() 8 >>> p = MyClass(5) >>> p.product() 5 >>> m.product() 8
>>> import math >>> math.cos(math.pi) -1.0
>>> math.cos(pi) Traceback (most recent call last): File "<stdin>", line 1, in <module> NameError: name 'pi' is not defined
>>> from math import cos, pi >>> cos(pi) -1.0
>>> from math import *
• Modules can contain any code
• Classes, functions, definitions, immediately executed code
• Can be imported in own namespace, or into the global namespace
>>> # My totally wicked function >>> def my_func(x, y=0.0, z=1.0): ... ”””This does some stuff. ... For example: >>> my_func(1.0, 3.0, 2.0) 8.0 Yep, it’s that good! ””” ... a = x + y ... b = a * z ... return b ...
• Comments before function, class, etc. are used to generate help
• “Docstrings” • preferred way of documenting code • can contain examples, which are automatically turned into tests! • See doctest module
>>> fname = ‘myfile.dat’
>>> f = file(fname) >>> lines = f.readlines() >>> f.close()
>>> f = file(fname) >>> firstline = f.readline() >>> secondline = f.readline()
>>> f = file(fname) >>> for l in f: >>> print l.split()[1] >>> f.close()
>>> outfname = ‘myoutput’ >>> outf = file(outfname, ‘w’) # second argument denotes writable >>> outf.write(‘My very own file\n’) >>> outf.close()
>>> sigma = 6.76/2.354 >>> print(‘sigma is %5.3f metres’%sigma) sigma is 2.872 metres >>> d = {‘bob’: 1.87, ‘fred’: 1.768} >>> for name, height in d.items(): ... print(‘%s is %.2f metres tall’%(name.capitalize(), height)) ... Bob is 1.87 metres tall Fred is 1.77 metres tall
>>> nsweets = range(100) >>> calories = [i * 2.345 for i in nsweets] >>> fout = file(‘sweetinfo.txt’, ‘w’) >>> for i in range(nsweets): ... fout.write(‘%5i %8.3f\n’%(nsweets[i], calories[i])) ... >>> fout.close()
• Also template system
>>> help(math)
>>> help(math.cos)
>>> a = [1, 2, 3] >>> help(a)
• Powerful help tools
• Every object, function, module, … can be inspected
• New 3.x branch is intentionally backwards incompatible
• Various improvements, removal of obsolete code, but annoying!
• print a ➔ print(a)
• 1/3 == 1//3 == 0 ➔ 1/3 == 1.0/3.0 == 1.333…, 1//3 == 0
• … various others …
• Version 2.6 contains most backward compatible changes, and can warn of usage (-3 switch) which would be an error in 3.x
• 2to3 (semi-)automated code translator
• Many useful modules not compatible with 3.x yet
• Use 2.6.x for now…
• Compromise between: • producing results quickly and • easy reusability and adaptation of code • code that can be quickly understood by others
• Comment clearly
• Use functions
• Use modules
• Consider ‘refactoring’ before code gets too messy
• Science, not software development
http://www.python.org/doc/
(note version numbers)
Part 2: Scientific programming in Python
An introduction to scientific programming with
Course webpage: http://stevenbamford.com/teaching/python_february_2010/
An introduction to scientific programming with
• Extra features required:
• fast, multidimensional arrays
• libraries of reliable, tested scientific functions
• plotting tools
• Lists ok for storing small amounts of one-dimensional data
• But, can’t use directly with arithmetical operators (+, -, *, /, …)
• Need efficient arrays with arithmetic and better multidimensional tools
• Numpy
• Similar to lists, but much more capable, except fixed size
>>> a = [1,3,5,7,9] >>> print(a[2:4]) [5, 7] >>> b = [[1, 3, 5, 7, 9], [2, 4, 6, 8, 10]] >>> print(b[0]) [1, 3, 5, 7, 9] >>> print(b[1][2:4]) [6, 8]
>>> import numpy
>>> import numpy >>> l = [[1, 2, 3], [3, 6, 9], [2, 4, 6]] # create a list >>> a = numpy.array(l) # convert a list to an array >>> print(a) [[1 2 3] [3 6 9] [2 4 6]] >>> a.shape (3, 3) >>> print(a.dtype) # get type of an array int32 >>> print(a[0]) # this is just like a list of lists [1 2 3] >>> print(a[1, 2]) # arrays can be given comma separated indices 9 >>> print(a[1, 1:3]) # and slices [6 9] >>> print(a[:,1]) [2 6 4]
>>> a[1, 2] = 7 >>> print(a) [[1 2 3] [3 6 7] [2 4 6]] >>> a[:, 0] = [0, 9, 8] >>> print(a) [[0 2 3] [9 6 7] [8 4 6]]
>>> b = numpy.zeros(5) >>> print(b) [ 0. 0. 0. 0. 0.] >>> b.dtype dtype(‘float64’) >>> n = 1000 >>> my_int_array = numpy.zeros(n, dtype=numpy.int) >>> my_int_array.dtype dtype(‘int32’)
>>> c = numpy.ones(4) >>> print(c) [ 1. 1. 1. 1. ]
>>> d = numpy.arange(5) # just like range() >>> print(d) [0 1 2 3 4]
>>> d[1] = 9.7 >>> print(d) # arrays keep their type even if elements changed [0 9 2 3 4] >>> print(d*0.4) # operations create a new array, with new type [ 0. 3.6 0.8 1.2 1.6]
>>> d = numpy.arange(5, dtype=numpy.float) >>> print(d) [ 0. 1. 2. 3. 4.]
>>> numpy.arange(3, 7, 0.5) # arbitrary start, stop and step array([ 3. , 3.5, 4. , 4.5, 5. , 5.5, 6. , 6.5])
>>> a = numpy.arange(4.0) >>> b = a * 23.4 >>> c = b/(a+1) >>> c += 10 >>> print c [ 10. 21.7 25.6 27.55]
>>> arr = numpy.arange(100, 200) >>> select = [5, 25, 50, 75, -5] >>> print(arr[select]) # can use integer lists as indices [105, 125, 150, 175, 195]
>>> arr = numpy.arange(10, 20 ) >>> div_by_3 = arr%3 == 0 # comparison produces boolean array >>> print(div_by_3) [ False False True False False True False False True False] >>> print(arr[div_by_3]) # can use boolean lists as indices [12 15 18]
>>> arr.sum() 45 >>> arr.mean() 4.5 >>> arr.std() 2.8722813232690143 >>> arr.max() 9 >>> arr.min() 0 >>> div_by_3.all() False >>> div_by_3.any() True >>> div_by_3.sum() 3 >>> div_by_3.nonzero() (array([2, 5, 8]),)
>>> arr = numpy.array([4.5, 2.3, 6.7, 1.2, 1.8, 5.5]) >>> arr.sort() # acts on array itself >>> print(arr) [ 1.2 1.8 2.3 4.5 5.5 6.7]
>>> x = numpy.array([4.5, 2.3, 6.7, 1.2, 1.8, 5.5]) >>> y = numpy.array([1.5, 2.3, 4.7, 6.2, 7.8, 8.5]) >>> numpy.sort(x) array([ 1.2, 1.8, 2.3, 4.5, 5.5, 6.7]) >>> print(x) [ 4.5 2.3 6.7 1.2 1.8 5.5] >>> s = x.argsort() >>> s array([3, 4, 1, 0, 5, 2]) >>> x[s] array([ 1.2, 1.8, 2.3, 4.5, 5.5, 6.7]) >>> y[s] array([ 6.2, 7.8, 2.3, 1.5, 8.5, 4.7])
• Most array methods have equivalent functions
• Ufuncs provide many element-by-element math, trig., etc. operations • e.g., add(x1, x2), absolute(x), log10(x), sin(x), logical_and(x1, x2)
• See http://numpy.scipy.org
>>> arr.sum() 45 >>> numpy.sum(arr) 45
• Array operations are implemented in C or Fortran
• Optimised algorithms - i.e. fast!
• Python loops (i.e. for i in a:…) are much slower
• Prefer array operations over loops, especially when speed important
• Also produces shorter code, often more readable
• User friendly, but powerful, plotting capabilites for python
• http://matplotlib.sourceforge.net/
• Once installed, to use type:
• Settings can be customised by editing ~/.matplotlib/matplotlibrc • default font, colours, layout, etc.
• Helpful website • many examples
>>> import pylab
• Suite of numerical and scientific tools for Python
• http://scipy.org/
Example modules:
• Optimisation / minimisation • http://docs.scipy.org/doc/scipy/reference/optimize.html
• Integration • http://docs.scipy.org/doc/scipy/reference/integrate.html
• Interpolation • http://docs.scipy.org/doc/scipy/reference/interpolate.html
• Statistics • http://docs.scipy.org/doc/scipy/reference/stats.html
• Image processing • http://docs.scipy.org/doc/scipy/reference/ndimage.html
• Python wrappers of GNU Scientific Library functions
• PyGSL: http://pygsl.sourceforge.net/
• GSL: http://www.gnu.org/software/gsl/
• Incomplete documentation for Python functions, but almost all of GSL is wrapped, so refer to GSL documentation.
• Most functionality implemented in SciPy • Try SciPy first, if you can’t find what you need try PyGSL • More comprehensive and sometimes more tested, but less ‘Pythonic’ • e.g. Monte Carlo integration
• http://rpy.sourceforge.net/
• Wraps R – a statistics analysis language • many advanced stats capabilities but quite specialised
• http://www.sagemath.org/
• Python-based mathematics software • replacement for Maple, Mathematica
• Planetary nebulae positions and velocities in edge-on galaxy NGC 891
• Need to: • read in data • make plot of positions, colour coded
by velocity • plot velocity histogram • fit projected rotation curve to full
dataset
Part 3: Python for observers
An introduction to scientific programming with
• http://web.mac.com/npirzkal/Scisoft
• Quick way to install many of the most used data reduction packages: • IRAF, PyRAF, MIDAS, Python and Python extensions and many more…
• http://www.stsci.edu/resources/software_hardware/pyraf/stsci_python
• Astronomy software provided by Space Telescope Science Institute
• PyFITS
• PyRAF
• MultiDrizzle
• Numdisplay
• STScI also provide STSDAS and TABLES packages for IRAF
• http://www.stsci.edu/resources/software_hardware/pyfits
• Read, write and manipulate all aspects of FITS files • extensions • headers • images • tables
• Low-level interface for details
• High-level functions for quick and easy use
>>> import pyfits >>> imgname = ‘data/2MASS_NGC_0891_K.fits’ >>> img = pyfits.getdata(imgname) >>> img array([[ 0. , 0. , 0. , ..., -999.00860596, -999.00860596, -999.00860596], [-999.00860596, -999.00860596, -999.00860596, ..., -999.00860596, -999.00860596, -999.00860596], [-999.00860596, -999.00860596, -999.00860596, ..., -999.00860596, -999.00860596, -999.00860596], ..., [-999.00860596, -999.00860596, -999.00860596, ..., -999.00860596, -999.00860596, -999.00860596], [-999.00860596, -999.00860596, -999.00860596, ..., -999.00860596, -999.00860596, -999.00860596], [-999.00860596, -999.00860596, -999.00860596, ..., -999.00860596, -999.00860596, -999.00860596]], dtype=float32) >>> img.mean() -8.6610549999999993 >>> img[img > -99].mean() 0.83546290095423026 >>> numpy.median(img) 0.078269213438034058
>>> x = 348; y = 97 >>> delta = 5 >>> print img[y-delta:y+delta+1, ... x-delta:x+delta+1].astype(numpy.int) [[ 1 1 1 1 1 0 0 0 1 0 -2] [ 2 2 4 6 7 7 4 3 1 0 -1] [ 1 4 11 24 40 40 21 7 2 0 0] [ 1 6 23 62 110 107 50 13 2 0 0] [ 2 7 33 91 158 148 68 15 3 0 0] [ 3 7 27 74 123 115 53 12 2 0 0] [ 2 4 12 32 54 51 24 5 1 0 0] [ 1 1 2 7 12 12 5 0 0 0 0] [ 0 0 0 1 2 2 1 0 0 1 0] [ 0 0 0 1 0 0 0 0 0 0 0] [ -1 0 1 0 0 0 0 0 0 0 0]]
• row = y = first index
• column = x = second index
• numbering runs as normal (e.g. in ds9) BUT zero indexed! x
y
>>> tblname = ‘data/N891PNdata.fits’ >>> d = pyfits.getdata(tblname) >>> d.names ('x0', 'y0', 'rah', 'ram', 'ras', 'decd', 'decm', 'decs', 'wvl', 'vel',
'vhel', 'dvel', 'dvel2', 'xL', 'yL', 'xR', 'yR', 'ID', 'radeg', 'decdeg', 'x', 'y')
>>> d.x0 array([ 928.7199707 , 532.61999512, 968.14001465, 519.38000488,… 1838.18994141, 1888.26000977, 1516.2199707 ], dtype=float32)
>>> d.field(‘x0’) # case-insensitive array([ 928.7199707 , 532.61999512, 968.14001465, 519.38000488,… 1838.18994141, 1888.26000977, 1516.2199707 ], dtype=float32)
>>> select = d.x0 < 200 >>> dsel = d[select] # can select rows all together >>> print dsel.x0 [ 183.05000305 165.55000305 138.47999573 158.02999878 140.96000671 192.58000183 157.02999878 160.1499939 161.1000061 136.58999634 175.19000244]
>>> h = pyfits.getheader(imgname) >>> print h SIMPLE = T BITPIX = -32 NAXIS = 2 NAXIS1 = 1000 NAXIS2 = 1200 BLOCKED = T / TAPE MAY BE BLOCKED IN MULTIPLES OF 2880 EXTEND = T / TAPE MAY HAVE STANDARD FITS EXTENSIONS BSCALE = 1. BZERO = 0. ORIGIN = '2MASS ' / 2MASS Survey Camera CTYPE1 = 'RA---SIN' CTYPE2 = 'DEC--SIN' CRPIX1 = 500.5 CRPIX2 = 600.5 CRVAL1 = 35.63922882 CRVAL2 = 42.34915161 CDELT1 = -0.0002777777845 CDELT2 = 0.0002777777845 CROTA2 = 0. EQUINOX = 2000. KMAGZP = 20.07760048 / V3 Photometric zero point calibration COMMENTC= 'CAL updated by T.H. Jarrett, IPAC/Caltech' SIGMA = 1.059334397 / Background Residual RMS noise (dn) COMMENT1= '2MASS mosaic image'
COMMENT2= 'created by T.H. Jarrett, IPAC/Caltech’ >>> h[‘KMAGZP’] 20.077600480000001 # Use h.items() to iterate through all header entries
>>> newimg = sqrt((sky+img)/gain + rd_noise**2) * gain >>> newimg[(sky+img) < 0.0] = 1e10
>>> hdr = h.copy() # copy header from original image >>> hdr.add_comment(‘Calculated noise image’)
>>> filename = ‘sigma.fits’
>>> pyfits.writeto(filename, newimg, hdr) # create new file
>>> pyfits.append(imgname, newimg, hdr) # add a new FITS extension
>>> pyfits.update(filename, newimg, hdr, ext) # update a file
# specifying a header is optional, # if omitted automatically adds minimum header
>>> import pyfits >>> import numpy as np
>>> # create data >>> a1 = numpy.array(['NGC1001', 'NGC1002', 'NGC1003']) >>> a2 = numpy.array([11.1, 12.3, 15.2])
>>> # make list of pyfits Columns >>> cols = [] >>> cols.append(pyfits.Column(name='target', format='20A',
array=a1)) >>> cols.append(pyfits.Column(name='V_mag', format='E', array=a2))
>>> # create HDU and write to file >>> tbhdu=pyfits.new_table(cols) >>> tbhdu.writeto(’table.fits’)
# these examples are for a simple FITS file containing just one # table or image but with a couple more steps can create a file # with any combination of extensions (see the PyFITS manual online)
>>> f = pyfits.open(tblname) >>> f.info() Filename: data/N891PNdata.fits No. Name Type Cards Dimensions Format 0 PRIMARY PrimaryHDU 4 () uint8 1 BinTableHDU 52 223R x 22C [E, E, E, E, E,
E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E]
>>> table = f[1] # data extension number 1 (can also use names)
>>> d = f[1].data # data, same as returned by pyfits.getdata() >>> h = f[1].header # header, same as returned by pyfits.getheader
()
>>> # make any changes >>> f.writeto(othertblname) # writes (with changes) to a new file
>>> f = pyfits.open(tblname, mode=‘update’) # to change same file >>> # make any changes >>> f.flush() # writes changes back to file >>> f.close() # writes changes and closes file
• http://www.stsci.edu/resources/software_hardware/pyraf
• Command line to replace cl, allows most normal IRAF commands and Python at same prompt
• Can use IRAF tasks in Python scripts instead of having to create awkward cl scripts (or worse SPP)
• Command and filename completion
• Edit line and access history easily (like ecl or bash)
• Use just as friendlier cl prompt or use Python whenever you want
• Transfer data between IRAF tasks and Python
• Use brackets for tasks when you want it to behave like Python
--> imstat 2MASS_NGC_0891_K.fits # IMAGE NPIX MEAN STDDEV MIN MAX 2MASS_NGC_0891_K.fits 1200000 -8.661 99.44 -1001. 7207.
--> fname = "data/2MASS_NGC_0891_K.fits" --> imstat fname # IMAGE NPIX MEAN STDDEV MIN MAX Error reading image fname ...
--> imstat(fname) # IMAGE NPIX MEAN STDDEV MIN MAX data/2MASS_NGC_0891_K.fits 1200000 -8.661 99.44 -1001. 7207.
• Many IRAF tasks create output images or tables on disk, or just print to screen, so don’t need to pass information back (see later for this)
stsdas # note can’t unload packages improject(sky_file_2D, sky_file_1D, projaxis=2, average='no') imcalc(sky_file_1D, sky_file_1D, 'im1*%f'%apwidthratio) # calculate effective gain and ron due to combined images gain = 1.91; ron = 5.41 gain_eff = gain * ncombine ron_eff = ron * sqrt(ncombine) # sig = sci + sky imcalc('%s,%s'%(sci_file, sky_file_1D), sig_file, 'im1+im2') # sig = sqrt(sig * gain + ron**2) / gain equation = 'sqrt(max(im1,0.0)/%(g)8.5f + %(r2)8.5f/%(g2)8.5f)' equation = equation%{'g': gain_eff, 'r2': ron_eff**2, 'g2': gain_eff**2} imcalc(sig_file, sig_file, equation)
• Use IRAF tasks in Python scripts
• Just import iraf object from pyraf module
from pyraf import iraf from glob import glob
images = glob(‘*sci.fits’)
for img in images: iraf.imstat(img) newimg = img.replace(‘sci’, ‘sig’) iraf.imcalc(img, newimg, ’sqrt(im1)')
• Can specify ‘default’ task parameters in neat fashion, instead of having to include on every call of a task
iraf.imstat.nclip = 3 iraf.imstat.lsigma = 5 iraf.imstat.usigma = 5
# now every time I use imstat it uses sigma clipping iraf.imstat(im1) iraf.imstat(im2) iraf.imstat(im3)
# can revert to task defaults by unlearning iraf.unlearn(‘imstat’) # note task name is a string
• Useful to make shortcuts
# shortcut for a long task name crrej = iraf.stsdas.hst_calib.wfpc.crrej crrej.mask = ‘mymask.fits’ crrej.sigma = 5
crrej(in1, out1) crrej(in2, out2) crrej(in3, out3)
• IRAF outputs lots of useful data to screen - how to get at it?
• Some tasks need user input - would be nice to automate these
• PyRAF defines extra parameters Stdout and Stdin for all tasks • Stdout can be a filename, file object, or 1 (one) to direct to a list • Stdin can be a filename, file object, or string
for img in images: # get a list with a string for each line of output output = iraf.imstat(img, format=‘no’,fields=‘midpt’,Stdout=1) # output can then be parsed as necessary to get at info med = float(output[0]) newimg = img.replace(‘sci’, ‘sub’) iraf.imcalc(img, newimg, ’im1 - %f’%med)
The End
An introduction to scientific programming with