Python Developer'sDaily Routine

Maxim Avanov (github.com/avanov)

Project structure

Project structure./chebit

|-- chebit

| -- __init__.py

-- setup.py

3

setup.pyfrom setuptools import setup, find_packages

setup(

name='Chebit',

version='0.0.1',

packages=find_packages(),

test_suite='tests',

tests_require=['pytest', 'coverage'],

install_requires=['Pyramid==1.5a2']

)

4

Bootstrapping$ python setup.py install

$ python setup.py develop

$ python setup.py test

$ python setup.py regtister sdist [bdist_egg] upload

5

Package

management

pip commands$ pip install Pyramid

$ pip install Pyramid --upgrade

$ pip install Pyramid==1.5a2 --upgrade

$ pip uninstall Pyramid

7

pip commands$ pip freeze

Babel==1.3

Chebit==0.0.1

CoffeeScript==1.0.8

Jinja2==2.7.1

Mako==0.9.0

...

8

pip-tools$ pip-review

requests==0.13.4 available (you have 0.13.2)

$ pip-review --auto

... <pip install output>

$ pip-review --interactive

requests==0.14.0 available (you have 0.13.2)

Upgrade now? [Y]es, [N]o, [A]ll, [Q]uit y

...

9

Cookiecutter• Generates initial project files from defined templates;

• File names are also templates;

• Transparently works with remote VCS.

10

Cookiecutter template./chebit-template

|-- cookiecutter.json

-- {{cookiecutter.project_name}}

|-- {{cookiecutter.project_name}}

| -- __init__.py

-- setup.py

11

cookiecutter.json{

"project_name": "myproject",

"project_version": "0.0.1"

}

12

Cookiecutter in action$ cookiecutter {project_template_path}

project_name (default is "myproject")? mynew

project_version (default is "0.0.1")? 0.1.1

$ tree ./mynew

./mynew

|-- mynew

| -- __init__.py

-- setup.py

13

We'd like to

isolate projectenvironments

virtualenv$ virtualenv ~/venv/{project_name}

$ source ~/venv/{project_name}/bin/activate

What if we need another Python version?$ virtualenv ~/venv/{project_name} -p python3

15

Still not happy

enough!

Introducing pyenv

pyenv - 5 steps to happiness$ git clone git://github.com/yyuu/pyenv.git ~/pyenv

$ echo 'export PYENV_ROOT="$HOME/pyenv"' >> ~/.bashrc

$ echo 'export PATH="$PYENV_ROOT/bin:$PATH"' >> ~/.bashrc

$ echo 'eval "$(pyenv init -)"' >> ~/.bashrc

$ exec $SHELL

18

pyenv - Rise and Shine!$ pyenv install -l

$ pyenv install 2.7.6

$ pyenv install 3.3.3

$ pyenv install pypy-2.2.1

$ pyenv rehash

19

The Happiness$ pyenv versions

$ pyenv version

$ pyenv local 2.7.6 3.3.3

$ pyenv global 3.3.3

$ pyenv shell pypy-2.2.1

$ pyenv whence 2to3

20

...

$ which python/home/ghostwriter/pyenv/shims/python

$ pythonPython 2.7.3 (87aa9de10f9c, Nov 24 2013, 18:48:13)[PyPy 2.2.1 with GCC 4.6.3] on linux2>>>>

21

virtualenv meets

pyenv

pyenv-virtualenv

$ git clone git://github.com/yyuu/pyenv-virtualenv.git ~/pyenv/plugins/pyenv-virtualenv

$ pyenv virtualenv pypy-2.2.1 ~/venv/{project_name}

23

pyenv-virtualenv in action$ pyenv virtualenvs

{project_name} (created from /home/ghostwriter/pyenv/versions/pypy-2.2.1)

$ source ~/pyenv/versions/{project_name}/bin/activate

24

Well done!

Now, what about

other components?

Virtualization

Vagrant• Written in Ruby

• Relies on VirtualBox

• Runs on Linux / OS X / Win

28

Juju• Written in Python / Go

• Relies on LXC

• Linux only

29

Docker• Written in Go

• Relies on LXC and AUFS

• Linux only

30

Vagrant's Success Story...1. Put a Vagrantfile into the project root

2. $ vagrant up

3. $ vagrant ssh

4. Profit!

31

Vagrantfile

Vagrant.configure("2") do |config|

config.vm.box = "project_name"

config.vm.box_url = "http://files.vagrantup.com/precise64.box"

config.vm.network :private_network, ip: "192.168.33.100"

end

32

...Vagrant's Success Story1. Put a Vagrantfile into the project root

2. $ vagrant up

3. $ vagrant ssh

4. Profit! ..???

5. Deployment

33

Deployment toolsChef (Ruby)

Puppet (Ruby)

Ansible (Python)

Salt (Python)

34

Ansible Success Story1. Specify an inventory file

2. Specify a playbook

3. $ ansible-playbook ./playbook.yml -i ./hosts

35

Ansible inventory[develop-vm]

192.168.33.100

36

Ansible playbook---

-

hosts: develop-vm

sudo: yes

tasks:

-

name: ensure apt cache is up to date

action: apt update_cache=yes

http://www.ansibleworks.com/docs/modules.html

37

Ansible meets Vagrant

$ ansible-playbook ./playbook.yml -i ./hosts -v -u vagrant -c paramiko --private-key=$HOME/.vagrant.d/insecure_private_key

or

$ vagrant provision

38

Ansible - Vagrant Integration# Vagrantfile

Vagrant.configure("2") do |config|

# ...

config.vm.provision :ansible do |ansible|

ansible.playbook = "./playbook.yml"

ansible.inventory_path = "./hosts"

ansible.verbose = "v"

end

end

39

Python & Web

Web Server Gateway InterfaceStandardized by PEP 333, PEP 3333

def application(environ, start_response):

start_response('200 OK', [('Content-Type', 'text/plain')])

yield 'Hello World\n'

41

WSGI Servers• CherryPy WSGI Server

• gunicorn

• tornado.wsgi

• uWSGI

• Waitress

http://nichol.as/benchmark-of-python-web-servers

42

Frameworks

Heavyweights• Django

• Twisted

• web2py

• Zope

44

Middleweights• CherryPy

• Pyramid

• Tornado

• TurboGears

45

Flyweights• Bottle

• Flask

• Webpy

46

Database Access

ORM• SQLAlchemy ORM

• Django ORM (Django only)

• Pony ORM

• Peewee

48

Something else• Native DB-API 2.0 (standardized with PEP 249)

• SQLAlchemy's raw queries

• mosql

49

SQL DDL Versioning & Data Migration• South (Django only)

• Alembic (SQLAlchemy)

• Pyrseas

50

Background task

processing

Celery• Out of the box solution for complex cases;

• Multiple brokers - RabbitMQ, Redis, RDBMS, SQS, MongoDB etc;

• Tasks scheduler;

• Results backends.

52

RQ (Redis Queue)• Designed to have a low barrier to enter;

• Uses Redis' Pub/Sub feature.

53

Process managers

Process managers• Supervisor

• Circus.io

55

Sentry

Testing tools• unittest

• py.test

• nose

• WebTest

• coverage

• tox

60

Thank you!