87
Install Python 2.5 or 2.6 (or pysqlite for 2.4) SQLAlchemy 0.5 1. easy_install, if needed wget http://peak.telecommunity.com/dist/ez_setup.py python ez_setup.py 2. easy_install sqlalchemy==0.5.8 http://old.utahpython.org/sqla2010/

PyCon 2010 SQLAlchemy tutorial

  • Upload
    jbellis

  • View
    129

  • Download
    3

Embed Size (px)

DESCRIPTION

Links to files are broken, sorry.

Citation preview

Page 1: PyCon 2010 SQLAlchemy tutorial

Install

● Python 2.5 or 2.6 (or pysqlite for 2.4)● SQLAlchemy 0.5

1. easy_install, if needed● wget http://peak.telecommunity.com/dist/ez_setup.py● python ez_setup.py

2. easy_install sqlalchemy==0.5.8

● http://old.utahpython.org/sqla2010/

Page 2: PyCon 2010 SQLAlchemy tutorial

Michael Bayer

Michael Bayer is a software architect in New York City and is the creator of SQLAlchemy.

http://techspot.zzzeek.org/@zzzeek

Page 3: PyCon 2010 SQLAlchemy tutorial

ORM 101 – the bad old days

c = db.cursor()sql = "SELECT * FROM users WHERE name = %s"c.execute(sql, (name,))user = User(*c.fetchone())

user.last_login_at = time.time()sql = "UPDATE users SET last_login_at = %s WHERE name = %s"c.execute(sql, (user.last_login_at, name0))c.commit()

Page 4: PyCon 2010 SQLAlchemy tutorial
Page 5: PyCon 2010 SQLAlchemy tutorial

ORM 101 – and there was light

session = Session()user = session.query(User).filter(name=name).one()

user.last_login_at = time.time()

session.commit()

Page 6: PyCon 2010 SQLAlchemy tutorial

The devil is in the details

● Compound WHERE clauses, subqueries, outer joins, sql functions, ...

● Eager/lazy loading● Support for legacy schemas● Inheritance● Conceptual integrity● Setup overhead● Database support

Page 7: PyCon 2010 SQLAlchemy tutorial

What SQLAlchemy is not

Page 8: PyCon 2010 SQLAlchemy tutorial

[Demolition photo]

Page 9: PyCon 2010 SQLAlchemy tutorial
Page 10: PyCon 2010 SQLAlchemy tutorial

Tough love?

“Disproving the myth of 'the best database layer is the one that makes the database invisible' is a primary philosophy of SA. If you don't want to deal with SQL, then there's little point to using a [relational] database in the first place.”

Page 11: PyCon 2010 SQLAlchemy tutorial

Technical excellence

● PK: multi-column is fine; mutable is fine; any data type is fine; doesn't have to be named “id”http://blogs.ittoolbox.com/database/soup/archives/primary-keyvil-part-i-7327

● Recognizes all database defaults instead of allowing a few special cases like “created_at”

● Doesn't make up its own query language● No XML● Introspection or define-tables-in-Python● Session/unit-of-work based● Migrations

Page 12: PyCon 2010 SQLAlchemy tutorial

Supported databases

PostgreSQLMySQLSQLiteFirebirdOracleMSSQLSybaseDB2InformixSAPDBMSAccess

Page 13: PyCon 2010 SQLAlchemy tutorial

Database dependence: a feature

● Performance– Functions, partial indexes, bitmap indexes,

partitioning, replication, ...● Features

– Views, arrays, recursive joins, full-text searching, ...

See also: http://powerpostgresql.com/Downloads/database_depends_public.sxi

Page 14: PyCon 2010 SQLAlchemy tutorial

Caching

Beaker integration is an example w/ the 0.6 distribution (currently in beta)

Page 15: PyCon 2010 SQLAlchemy tutorial

Questions

Page 16: PyCon 2010 SQLAlchemy tutorial

Today's agenda: fundamentals

Data Mapper vs Active RecordSA FundamentalsMapping basicsQueriesSessions & the identity mapRelationship lifecycleBackrefs

Page 17: PyCon 2010 SQLAlchemy tutorial

Agenda 2: More ORM details

Multi-object queriesOne-to-oneMany-to-manyRelation queriesEager/lazy loadingTransactionsExtensions and related projects

Page 18: PyCon 2010 SQLAlchemy tutorial

Agenda 3: Extensions and related projects

MigrateFormAlchemySqlSoupElixirz3c.sqlalchemy

Page 19: PyCon 2010 SQLAlchemy tutorial

Two ORM patterns

Active RecordData Mapper

Page 20: PyCon 2010 SQLAlchemy tutorial

Active Record

Page 21: PyCon 2010 SQLAlchemy tutorial

Data Mapper

Page 22: PyCon 2010 SQLAlchemy tutorial

SQLAlchemy supports both

● Declarative plugin for common simple situations

● Full data mapper power when you need it

Page 23: PyCon 2010 SQLAlchemy tutorial

Tables for this tutorial

usersaddressesordersorderitemskeywordsitemkeywords

Page 24: PyCon 2010 SQLAlchemy tutorial

Tables

users = Table('users', metadata, Column('user_id', Integer, primary_key = True), Column('name', String(40)))

users = Table('users', metadata, autoload=True)

users = Table('users', metadata, autoload=True, Column('name', String(40), default='Jonathan'))

Page 25: PyCon 2010 SQLAlchemy tutorial
Page 26: PyCon 2010 SQLAlchemy tutorial

Legacy columns

ack = Table('ACK110030', metadata, Column('ITMNUMBER', Integer, primary_key=True, key='id'), Column('MNFCENTERLC_I', Integer, ForeignKey('MFC43222.id'), key='manufacturing_center_id'), ...)

Page 27: PyCon 2010 SQLAlchemy tutorial

Setup from scratch

engine = create_engine('sqlite:///:memory:', echo=True)metadata = MetaData()metadata.bind = engineSession = sessionmaker(bind=engine)

from sqlalchemy.ext.declarative import declarative_baseBase = declarative_base()

Page 28: PyCon 2010 SQLAlchemy tutorial

Quick setup

from tutorial_tables import *create()data()

http://old.utahpython.org/sqla2010/

Page 29: PyCon 2010 SQLAlchemy tutorial

Table + mapped class together

class Order(Base): __tablename__ = 'orders' order_id = Column(Integer, primary_key=True) user_id = Column(Integer, ForeignKey(users.c.user_id)) description = Column('description', String(50)) isopen = Column(Integer, ColumnDefault(1))

Page 30: PyCon 2010 SQLAlchemy tutorial

Full data mapper pattern

orders = Table('orders', metadata, Column('order_id', Integer, ...), Column('user_id', Integer, ForeignKey(users.c.user_id)), Column('description', String(50)), Column('isopen', Integer, ColumnDefault(1)),

class Order(object): pass

mapper(Order, orders)

Page 31: PyCon 2010 SQLAlchemy tutorial

The way we will do mapping

class Order(Base): __table__ = orders

Page 32: PyCon 2010 SQLAlchemy tutorial

Querying

session = Session()q = session.query(Order)print q

.all

.get

.first, .one

Page 33: PyCon 2010 SQLAlchemy tutorial

Query modification

q = session.query(Order)

.filter

.filter_by

.order_by [desc, asc]

.limit, .offset

Page 34: PyCon 2010 SQLAlchemy tutorial

Operators

== >= <= > <

~ | &not_ or_ and_

in_

between like startswith endswith

Page 35: PyCon 2010 SQLAlchemy tutorial

Some examples

q = session.query(Order)

q.filter_by(user_id=7).order_by(Order.isopen).first()q.filter(Order.description.like('order%')).all()q.filter((Order.user_id==7) | (Order.isopen==1)).all()q.filter(or_(Order.user_id==7, Order.isopen==1)).all()

Page 36: PyCon 2010 SQLAlchemy tutorial

Slicing: limit/offset sugar

q = session.query(Order)q.limit(1).offset(2)

q[2:3]

Page 37: PyCon 2010 SQLAlchemy tutorial

Questions

Page 38: PyCon 2010 SQLAlchemy tutorial

Exercise

● Map the orderitems table to an OrderItem class

● Get a list of all OrderItems– Where they belong to order #3– ... or the item name is “item 1”– ... ordered by item name

(Now would be a good time to look at tutorial_samples.py)

Page 39: PyCon 2010 SQLAlchemy tutorial

clear_mappers()

Page 40: PyCon 2010 SQLAlchemy tutorial

Creating, updatingo = Order()o.user_id = 7o.description = 'order 6'session.add(o)o.order_id is Nonesession.commit()o.order_id == 6o.description = 'order B'session.commit()

session.delete(o)session.commit()

Page 41: PyCon 2010 SQLAlchemy tutorial

Scoped (“smart”) sessions

Session = scoped_session( sessionmaker(autoflush=True, autocommit=False))

assert Session() == Session()

Page 42: PyCon 2010 SQLAlchemy tutorial

Scoped sessions 2

Base = declarative_base(metadata=Session.metadata)class Order(Base): ...

o = Order()o.user_id = 7o.description = 'order 6'session.commit()

Page 43: PyCon 2010 SQLAlchemy tutorial

Direct updates, deletes

orders.update(orders.c.order_id==2).execute(isopen=1)orders.delete(orders.c.order_id==2).execute()

SQL layer alert!

Page 44: PyCon 2010 SQLAlchemy tutorial

One-to-many relations

class User(Base): orders = relation(Order, order_by=[Order.order_id])u = session.query(User).first()print u.orders

Page 45: PyCon 2010 SQLAlchemy tutorial

Editing collectionso = Order(description='An order')u.orders.append(o)session.commit()

Page 46: PyCon 2010 SQLAlchemy tutorial

Why sessions are your friends

Some ORMs rely on explicit save

More convenient, less error-prone to let ORM track dirty objects

u = User.get(1)u.orders[0].description = 'An order'u.save() # not real SA code# doh! orders[0] was not saved!

Page 47: PyCon 2010 SQLAlchemy tutorial

Identity map

Rows with same PK get mapped to same object (per-session)Limited caching for get()Only for get()

Page 48: PyCon 2010 SQLAlchemy tutorial

Managing the identity map

sesion.query(cls).populate_existing()session.expire(obj)session.refresh(obj)session.expunge(obj)expunge_all, expire_all

Page 49: PyCon 2010 SQLAlchemy tutorial

Questions

Page 50: PyCon 2010 SQLAlchemy tutorial

Exercise

Load the user named 'jack' (lowercase)Remove his first orderSave changes to the db

Page 51: PyCon 2010 SQLAlchemy tutorial

Fun with collectionsu = session.query(User).filter_by(name='jack').one()u.orders = u.orders[1:]session.commit()

>>> session.query(Order).get(1)Order(order_id=1,user_id=None,...)

Page 52: PyCon 2010 SQLAlchemy tutorial

Two solutions

o = u.orders[0]>>> oOrder(order_id=1,user_id=7,description=u'order 1',isopen=0)session.delete(o)session.commit()

>>> u.orders[Order(order_id=3,...), Order(order_id=5,...)]

Why does it make sense for this to work differently than the previous example?

Page 53: PyCon 2010 SQLAlchemy tutorial

#2: delete-orphanclass User(Base): __table__ = users orders = relation(Order, cascade="all, delete-orphan", order_by=[Order.order_id])

u = session.query(User).get(7)u.orders = u.orders[1:]session.commit()

>>> session.query(Order).get(1) is NoneTrue

Page 54: PyCon 2010 SQLAlchemy tutorial

Questions

Page 55: PyCon 2010 SQLAlchemy tutorial

Exercise

def user_for_order(order): session = Session.object_session(order) return ?

Page 56: PyCon 2010 SQLAlchemy tutorial

Backrefs

class User(Base): __table__ = users orders = relation(Order, backref='user', order_by=[orders.c.order_id])

o = session.query(Order).first()o.user

Page 57: PyCon 2010 SQLAlchemy tutorial

That's it for fundamentals

Multi-object queriesOne-to-oneMany-to-manyRelation queriesEager/lazy loadingTransactionsExtensions and related projects

Page 58: PyCon 2010 SQLAlchemy tutorial

Exercise

List all users who have open orders (isopen is nonzero)

Page 59: PyCon 2010 SQLAlchemy tutorial

A taste of advanced querying

q = session.query(User).join(User.orders)q.filter(Order.isopen==1).all()

All orders with an open order:

Page 60: PyCon 2010 SQLAlchemy tutorial

Selecting multiple classes

q = session.query(User, Order) # cartesian join!q = q.join(User.orders)# or!q = q.filter(User.user_id==Order.user_id)

u, o = q.filter(Order.isopen==1).first()

What if we want the user and the order? (efficiently)

Page 61: PyCon 2010 SQLAlchemy tutorial

Dropping down to SQL

sql = """select u.*from users uwhere u.user_id in ( select user_id from orders where isopen = 1)"""session.query(User).from_statement(sql)

sql = """select u.*, o.*from users u join orders o on (u.user_id = o.user_id)where o.isopen = 1"""session.query(User, Order).from_statement(sql)

Page 62: PyCon 2010 SQLAlchemy tutorial

Dropping down a little less

q = session.query(User, Order)q = q.join(User.orders)q.filter("orders.isopen = 1").all()

Page 63: PyCon 2010 SQLAlchemy tutorial

Exercise

In a single query, select the users and orders where the order description is like 'order%'

Page 64: PyCon 2010 SQLAlchemy tutorial

One to one

class Address(Base): __table__ = addresses

class User(Base): orders = relation(Order, order_by=[orders.c.order_id]) address = relation(Address)

How does SQLA know to treat these differently?

Page 65: PyCon 2010 SQLAlchemy tutorial

Many to many

class Keyword(Base): __table__ = keywords

class Item(Base): __table__ = orderitems keywords = relation(Keyword, secondary=itemkeywords)

Page 66: PyCon 2010 SQLAlchemy tutorial

Relation queries

user = session.query(User).get(7)q = session.query(Order)q.filter(Order.user==user).all()# q.filter(Order.user_id==user.user_id).all()

Page 67: PyCon 2010 SQLAlchemy tutorial

Relation queries 2

q = session.query(Order)

q.filter(Order.user.has(name='jack')).all()q.filter(Order.user.has((User.name=='jack') | (User.user_id >= 9))).all()# q.filter(Order.user_id== select([users.c.user_id], users.c.name=='jack')).all()# q.filter(Order.user_id== select([users.c.user_id], (users.c.name=='jack') | (users.c.user_id >= 9))).all()

Page 68: PyCon 2010 SQLAlchemy tutorial

Relation queries 3

q = session.query(User)

q.filter(User.orders.any(Order.isopen > 0)).all()q.filter(User.orders.any( Order.description.like('order%'))).all()# q.filter(User.user_id.in_( select([orders.c.user_id], orders.c.isopen > 0))).all()# q.filter(User.user_id.in_( select([orders.c.user_id], orders.c.description.like('order%')))).all()

Page 69: PyCon 2010 SQLAlchemy tutorial

Relation queries 4

Just need a raw EXISTS clause? Use .any() or .has() without extra parameters.

Page 70: PyCon 2010 SQLAlchemy tutorial

Relation queries 5

Keyword = session.query(Keyword).filter_by(name='red').one()q = session.query(Item)

q.filter(Item.keywords.contains(keyword)).all()# q.filter(Item.item_id.in_( select([itemkeywords.c.item_id], itemkeywords.c.keyword_id ==keyword.keyword_id))).all()

Page 71: PyCon 2010 SQLAlchemy tutorial

Exercise

Retrieve all users that do not have any orders.

Page 72: PyCon 2010 SQLAlchemy tutorial

Breathe Easy

That was our last exercise!

Page 73: PyCon 2010 SQLAlchemy tutorial

Eager loading

class User(Base): orders = relation(Order, order_by=[orders.c.order_id], lazy=False)# orq = session.query(User).options(eagerload('orders'))

# also lazyload, noload

Page 74: PyCon 2010 SQLAlchemy tutorial

Transactions are simple

● session: autocommit=False– this is the default– commit() / rollback() manually

● autocommit=True– each flush() also commits

Page 75: PyCon 2010 SQLAlchemy tutorial

Other transaction features

● begin_nested● manual transaction management at the

Connection level

Page 76: PyCon 2010 SQLAlchemy tutorial

__init__

class Order(Base): __table__ = orders def __init__(self): self.foo = []

@reconstructor def loaded(self): self.foo = []

__init__ is for object creation, not loadinguse @reconstructor

Page 77: PyCon 2010 SQLAlchemy tutorial

Questions

Page 78: PyCon 2010 SQLAlchemy tutorial

Related projects

MigrateFormAlchemySqlSoupElixirz3c.sqlalchemy

Page 79: PyCon 2010 SQLAlchemy tutorial

Migrate

# one-time setup

migrate create path/to/upgradescripts "comment"

migrate manage dbmanage.py --repository=path/to/upgradescripts –url=db-connection-url

./dbmanage.py version_control

# repeat as necessary:

./dbmanage.py script_sql sqlite# edit script

./dbmanage.py upgrade

Page 80: PyCon 2010 SQLAlchemy tutorial

FormAlchemy

order1 = session.query(Order).first()

from formalchemy import FieldSet fs = FieldSet(order1) print fs.render()

Page 81: PyCon 2010 SQLAlchemy tutorial

FormAlchemy, cont.

from formalchemy import Grid orders = session.query(Order).all() g = Grid(Order, orders) print g.render()

Page 82: PyCon 2010 SQLAlchemy tutorial

SqlSoup

>>> from sqlalchemy.ext.sqlsoup import SqlSoup>>> db = SqlSoup(metadata)

>>> db.users.filter(db.users.user_id < 10).all()[MappedUsers(user_id=7,name='jack'), MappedUsers(user_id=8,name='ed'), MappedUsers(user_id=9,name='fred')]

>>> db.users.first()MappedUsers(user_id=7,name='jack')>>> _.ordersTraceback (most recent call last): File "<stdin>", line 1, in <module>AttributeError: 'MappedUsers' object has no attribute 'orders'

Page 83: PyCon 2010 SQLAlchemy tutorial

SqlSoup 2

>>> db.users.relate('orders', db.orders)>>> db.users.first().orders[MappedOrders(...)]

>>> db.users.filter(db.users.orders.any()).all()[MappedUsers(...)]

Page 84: PyCon 2010 SQLAlchemy tutorial

SqlSoup 3

s = select([func.count('*')], users.c.user_id==orders.c.user_id, from_obj=[orders], scalar=True)s2 = select([users, s.label('order_count')]).alias('users_with_count')db.users_with_count = db.map(s2)

Page 85: PyCon 2010 SQLAlchemy tutorial

Elixir

class Person(Entity): has_field('name', Unicode) acts_as_taggable() ... some_person_instance.add_tag('cool') ... cool_people = Person.get_by_tag('cool')

http://cleverdevil.org/computing/52/

Page 86: PyCon 2010 SQLAlchemy tutorial

Resources

irc://irc.freenode.net/#sqlalchemySQLA mailing listMigrate, FormAlchemy, Elixir

Page 87: PyCon 2010 SQLAlchemy tutorial

Final Questions