British Atmospheric Data Centre Python Eggs. British Atmospheric Data Centre Overview Part I: Installers What are Eggs? What Eggs can do for me?

British Atmospheric Data Centre

Python Eggs


Overview Part I: Installers

What are Eggs? What Eggs can do for me? Package installation with Eggs

Part II: Developers Component overview Package compatibility Developing with Eggs

Part III: Demo


What are Python Eggs?

Eggs = +

JARs CPAN

+ Backwards Compatibility


Eggs ≈ JARs

.egg zip archive Drop into sys.path pkg_resources API

What about Dependencies? Scripts? C Extensions?

EGG-INFO

package_1

package_2

code

data

_____________.egg


Eggs ≈ CPAN

$ easy_install MyPackage>=2.1... find MyPackage v2.1 ...

... download MyPackage v2.1 ...

... download MyPackage dependencies ...

... build all ...

... install all ...

$ python

>>> import MyPackage

>>>


Python Cheeseshop



easy_install examples

$ easy_install MyPackage

$ easy_install MyPackage==3.2

$ easy_install -f http://example.com MyPackage

$ easy_install --upgrade MyPackage

$ easy_install –d temp_eggs MyPackage-prealpha

$ easy_install $SROOT/MyPackage/trunk#egg=MyPackage-dev

$ easy_install -m MyPackage==3.2a1


Installing setuptoolsrecipe 1: globally

Ensure distutils is available Ensure you can write python installation

$ wget http://peak.telecommunity.com/dist/ez_setup.py

$ python ez_setup.py


Installing setuptoolsrecipe 2: using PYTHONPATH Ensure distutils is available In ~/.profile

export PYTHONPATH=$HOME/lib/pythonexport PATH=$HOME/bin:$PATH

In ~/.pydistutilsrc

[install]install_lib = ~/lib/pythoninstall_scripts = ~/bin

$ wget http://peak.telecommunity.com/dist/ez_setup.py$ python ez_setup.py


Key Advantages

Self contained Try new software easily

install / test / throw away Track development

select version download from SVN

Supports plug-in architectures


Part IIDevelopers


Egg components<<package>>pkg_resources

dependenciesresourcesmetadatanamespaces

<<package>>setuptools

build eggs

<<package>>distutils

<<script>>easy_install

finddownloadinstallupgrade<<egg>>

setuptools

<<script>>ez_setup.py


Distutils

$ python setup.py install

$ python setup.py install –prefix=$HOME

$ python setup.py build

$ sudo python setup.py install

$ python setup.py build –build-opt install


Distutils

Problems Insufficient documentation Easier to shortcut than use properly

Consequences Inconsistent setup.py interface Monolithic frameworks


CDAT


cdat-lite / vcs-lite

$ URL=http://home.badc.rl.ac.uk/spascoe/ndg_eggs$ easy_install –f $URL vcs-lite


Matplotlib


Matplotlib

easy_install numpy

easy_install matplotlib

easy_install basemap

easy_install –f $SVN basemap


M2Crypto

$ easy_install M2Crypto

No way to use the –openssl option!

setup.py's key mistakes Parses the –openssl argument outside distutils. Sets include_dirs even if using default location.


It will probably work if you

Stick to the distutils framework! including

argument parsing install locations

use __file__ or __name__ to find resources


Setuptools: preliminariesimport ez_setup

ez_setup.use_setuptools()

from setuptools import setup, find_packages

setup(

name = 'MyPackage',

version = '1.0',

...

packages = find_packages('lib',

exclude=["test_*"]),

package_dir = {'' : 'lib'},

...

)


Setuptools: dependenciessetup(

...

install_requires = [

'MyDep1>=0.3', 'MyDep2==1.2rc2',

'MyDep3==dev'

],

dependency_links = [

'http://www.example.com/mydep2',

'http://www.example.com/svn/mydep3/trunk#egg=MyDep3-dev'

],

...

)


Setuptools: resources

setup(

...

zip_safe = False,

include_package_data = True,

exclude_package_data = {'': ['README.txt']},

...

)


Setuptools: scripts

setup(

...

entry_points = {

'console_scripts': [

'foo = my_package.some_module:main_func',

'bar = other_module:some_func',

],

'gui_scripts': [

'baz = my_package_gui.start_func',

]},

...

)


Setuptools: and more

namespace packages entry points extras SVN hooks unit tests


pkg_resources API

#udunits_name=sys.prefix+'/lib/python'+version+'/site-packages/unidata/udunits.dat'

import pkg_resourcesudunits_name = pkg_resources.resource_filename('unidata', 'udunits.dat')

import pkg_resourcespkg_resources.require('ZSI==2.0rc1')


Using Eggs in NDG

Stripping out CDAT components cdat-lite, vcs-lite

Splitting the ndg package into multiple eggs namespace packages

Managing multiple versions ZSI

Shared code in ndg_util.egg Component bundles


Part III: Demo

A WSGI compliant OpeNDAP / WMS server in exactly 0 lines of python


#!/bin/sh

# Log commands to stdoutset -x

# Setup staging arearm -fr eggs myserver demo.logmkdir eggsexport PYTHONPATH=$PWD/eggsexport PATH=$PYTHONPATH:$PATH

DAPSVN=http://pydap.googlecode.com/svn/trunk/

# Bootstrap setuptoolspython ez_setup.py -d eggs >>demo.log 2>&1

# Install pydap prerequisiteseasy_install -d eggs -f http://www.pythonware.com/products/pil \ numpy Imaging >>demo.log 2>&1

# Install pydap and pluginseasy_install -d eggs \ dap[server] dap.plugins.netcdf dap.responses.wms \ $DAPSVN/responses/html >>demo.log 2>&1

# Create the server projectpaster create -t dap_server myserver project=My-Server >>demo.log 2>&1

# Move data to server directorycp *.nc myserver/data >>demo.log 2>&1

# Start the serverpaster serve myserver/server.ini

Demo script



[server:main]use = egg:Paste#http# Change to 0.0.0.0 to make publichost = 127.0.0.1port = 8080

[filter-app:main]use = egg:Paste#httpexceptionsnext = cascade

[composit:cascade]use = egg:Paste#cascadeapp1 = staticapp2 = pydapcatch = 404

[app:static]use = egg:Paste#staticdocument_root = %(here)s/data

[app:pydap]use = egg:dapname = My-Serverroot = %(here)s/dataverbose = 0template = %(here)s/template

Demo paste.deploy configuration




The End


References

http://peak.telecommunity.com/DevCenter/PythonEggs http://peak.telecommunity.com/DevCenter/EasyInstall http://peak.telecommunity.com/DevCenter/setuptools http://peak/telecommunity.com/DevCenter/PkgResources

http://mail.python.org/mailman/listinfo/distutils-sig/

Documents

British Atmospheric Data Centre Python Eggs. British Atmospheric Data Centre Overview Part I: Installers What are Eggs? What Eggs can do for me?