Upload
jannis-leidel
View
7.981
Download
0
Tags:
Embed Size (px)
Citation preview
6How I learned to stop
worrying & love Python Packaging
1) The past of Python packaging2) Common pitfalls and gotchas3) Everyday software development4) The future of Python packaging
6
- Python packageA directory with Python files (module)and a __init__.py file
- ReleaseA version of a specific software, resultof a development cycle, e.g. “Django 1.3”
- DistributionSource or binary form of a release,e.g. zipped tarball or Windows installer
Ambiguous terms
1The past of Python packaging, a brief historical overview
“[..] a standard mechanism for building, distributing, and installing Python modules – or, more realistically, multi-module distributions.”
Distutils history
Greg Ward
- creates distributions
- installs to library directory
- builds C extensions
- processes documentation
- PEP 241 (2001)Metadata in PKG-INFO files added to with package distributions
- PEP 301 (2002)Package Index and Trove classifiers added and register command
- PEP 314 (2003)Metadata 1.1 with license, platform, download URL, dependencies fields
Distutils enhancements
- PEP 345 (2005-today)Metadata 1.2, better dependencies
- PEP 376 (2009-today)Database of Installed Python packages
- PEP 386 (2009-today)Sane version comparison module and better version handling on PyPI
Distutils enhancements
2Common pitfalls and gotchas of packaging
Python software
from distutils.core import setup
setup( name='MyApp', version='0.1.0', description='My app that does things', author='John Doe', author_email='[email protected]', license='BSD', packages=['myapp'], package_data = { 'myapp': [ 'templates/*.html', ], },)
The standard setup.py
Long description
Rendered on PyPI with docutilsreStructuredText renderer
import codecsdef readme(filename): file = codecs.open(filename, 'utf-‐8') return unicode(file.read())
setup( # ... long_description = readme('README.rst'),)
Lists all additional, non-Python files of a release to be installed from a distribution
setup( # ... package_data = { 'myapp': [ 'static/myapp/*/*', 'templates/myapp/*', 'locale/*/LC_MESSAGES/*' ], },)
package_data
Template for the “MANIFEST” file thatlists all files to be put in a distribution
include README.rstrecursive-‐include docs *.txtrecursive-‐include myapp/locale *.mo *.porecursive-‐include myapp/static *.css *.pngrecursive-‐include myapp/templates *.html
MANIFEST.in
- 0.18 “Catty And The Major”
- .000001
- “unreleased.uno!cialdev”
Don’t do that.
- Use PEP 386 for formal versions, e.g.“1.0.1” or “0.5a4” etc
- Use release names in any prose documentation and changelogs,e.g. super hero names
Use sane versions, dammit!
3 Using packaging in everyday’s software
development
- Install binary packages globally using your operatings system’s package management, e.g. database adapters, PIL, lxml
- Vendorize packages that require modifications or short term patches
- Install often updated and your project specific packages locally, e.g. Django, pytz, your own app
System package management or not?
- Isolated Python environments
- Create by topic, e.g. by “branch”, “client”, “milestone”, “staging/prod” etc.
- Use virtualenv-wrapper for even more shell helpers
- pip included in every virtual environment
virtualenv
- Sane installation and uninstallation
- freezing and unfreezing of requirements
$ server1:~ pip freeze > dev-‐deps.txt
$ server2:~ pip install -‐r dev-‐deps.txt
- VCS integration for quick and dirty use
- PyPI mirror support (PEP 381), e.g. d.pypi.python.org
pip
- HTTP server with directory index feature, e.g. Apache’s DirectoryIndex
- PyPI clone Chishop
pip install -‐f http://localhost/dists/
pip install -‐i http://pypi.corp.local/
- Use ~/.pip/pip.conf
[install]
index-‐url = http://pypi.corp.local/
Simple own package index
- Use ad-hoc HTTP server
$ python -‐m SimpleHTTPServer 8000
- .pydistutils.cfg sdist dir
[sdist]
dist-‐dir = /var/www/dists
- Enables pip download-cache
[global]
download-‐cache = ~/.pip-‐downloads
Local index
4The future
of packagingwith Distutils
- toolbox with reference implementation for PEP 345, PEP 376, PEP 386
- the standalone pysetup tool thatinstalls and uninstalls distributions(e.g. “pysetup install Django”)
- metadata available outside of Python file in setup.cfg
- already in Python 3 trunk as “packaging”
What is distutils2?
- defined requirements depending on environment
Requires-‐Dist: bar; python_version == '2.4' or python_version == '2.5'
Requires-‐Python: >=2.5,<3
- Obsoleting other releases
Obsoletes-‐Dist: OtherProject (<3.0)
Other distutils2 features
- “pysetup create” for conversion setup.py to setup.cfg for forward compatibility
- “pysetup generate-setup” for conversion setup.cfg to setup.py for backwards compatibility
Moving from distutils to distutils2
[metadata]name = myappversion = 0.1author = John Doeauthor-‐email = [email protected] = My awesome appdescription-‐file = README.rsthome-‐page = http://johndoe.com/p/myapp/project-‐url: Repository, http://code.johndoe.com/myapp/classifier = Development Status :: 3 -‐ AlphaLicense :: OSI Approved :: MIT License
[files]packages = myappextra_files = setup.py READMEresources = etc/settings.py.dist {confdir}/myapp
Example distutils2 setup.cfg
- METADATA
- RECORDa list of installed files (CSV formatting)
- RESOURCESlist of non-python files
- INSTALLERname of the installer
- REQUESTEDexists if distribution wasn’t installed as dependency
Other files that are shipped with distutils2
1) Keep your setup.py files simple2) Follow the standards3) Use virtualenv/pip4) Look out for distutils2/packaging5) In doubt, read the
“Hitchhiker’s Guide to Packaging”
6
d @enn_io
oW Jannis Leidel
Thanks!
Questions?
Don‘t forget to
while we’re in Amsterdam!