Upload
jbellis
View
129
Download
3
Tags:
Embed Size (px)
DESCRIPTION
Links to files are broken, sorry.
Citation preview
Install
● Python 2.5 or 2.6 (or pysqlite for 2.4)● SQLAlchemy 0.5
1. easy_install, if needed● wget http://peak.telecommunity.com/dist/ez_setup.py● python ez_setup.py
2. easy_install sqlalchemy==0.5.8
● http://old.utahpython.org/sqla2010/
Michael Bayer
Michael Bayer is a software architect in New York City and is the creator of SQLAlchemy.
http://techspot.zzzeek.org/@zzzeek
ORM 101 – the bad old days
c = db.cursor()sql = "SELECT * FROM users WHERE name = %s"c.execute(sql, (name,))user = User(*c.fetchone())
user.last_login_at = time.time()sql = "UPDATE users SET last_login_at = %s WHERE name = %s"c.execute(sql, (user.last_login_at, name0))c.commit()
ORM 101 – and there was light
session = Session()user = session.query(User).filter(name=name).one()
user.last_login_at = time.time()
session.commit()
The devil is in the details
● Compound WHERE clauses, subqueries, outer joins, sql functions, ...
● Eager/lazy loading● Support for legacy schemas● Inheritance● Conceptual integrity● Setup overhead● Database support
What SQLAlchemy is not
[Demolition photo]
Tough love?
“Disproving the myth of 'the best database layer is the one that makes the database invisible' is a primary philosophy of SA. If you don't want to deal with SQL, then there's little point to using a [relational] database in the first place.”
Technical excellence
● PK: multi-column is fine; mutable is fine; any data type is fine; doesn't have to be named “id”http://blogs.ittoolbox.com/database/soup/archives/primary-keyvil-part-i-7327
● Recognizes all database defaults instead of allowing a few special cases like “created_at”
● Doesn't make up its own query language● No XML● Introspection or define-tables-in-Python● Session/unit-of-work based● Migrations
Supported databases
PostgreSQLMySQLSQLiteFirebirdOracleMSSQLSybaseDB2InformixSAPDBMSAccess
Database dependence: a feature
● Performance– Functions, partial indexes, bitmap indexes,
partitioning, replication, ...● Features
– Views, arrays, recursive joins, full-text searching, ...
See also: http://powerpostgresql.com/Downloads/database_depends_public.sxi
Caching
Beaker integration is an example w/ the 0.6 distribution (currently in beta)
Questions
Today's agenda: fundamentals
Data Mapper vs Active RecordSA FundamentalsMapping basicsQueriesSessions & the identity mapRelationship lifecycleBackrefs
Agenda 2: More ORM details
Multi-object queriesOne-to-oneMany-to-manyRelation queriesEager/lazy loadingTransactionsExtensions and related projects
Agenda 3: Extensions and related projects
MigrateFormAlchemySqlSoupElixirz3c.sqlalchemy
Two ORM patterns
Active RecordData Mapper
Active Record
Data Mapper
SQLAlchemy supports both
● Declarative plugin for common simple situations
● Full data mapper power when you need it
Tables for this tutorial
usersaddressesordersorderitemskeywordsitemkeywords
Tables
users = Table('users', metadata, Column('user_id', Integer, primary_key = True), Column('name', String(40)))
users = Table('users', metadata, autoload=True)
users = Table('users', metadata, autoload=True, Column('name', String(40), default='Jonathan'))
Legacy columns
ack = Table('ACK110030', metadata, Column('ITMNUMBER', Integer, primary_key=True, key='id'), Column('MNFCENTERLC_I', Integer, ForeignKey('MFC43222.id'), key='manufacturing_center_id'), ...)
Setup from scratch
engine = create_engine('sqlite:///:memory:', echo=True)metadata = MetaData()metadata.bind = engineSession = sessionmaker(bind=engine)
from sqlalchemy.ext.declarative import declarative_baseBase = declarative_base()
Quick setup
from tutorial_tables import *create()data()
http://old.utahpython.org/sqla2010/
Table + mapped class together
class Order(Base): __tablename__ = 'orders' order_id = Column(Integer, primary_key=True) user_id = Column(Integer, ForeignKey(users.c.user_id)) description = Column('description', String(50)) isopen = Column(Integer, ColumnDefault(1))
Full data mapper pattern
orders = Table('orders', metadata, Column('order_id', Integer, ...), Column('user_id', Integer, ForeignKey(users.c.user_id)), Column('description', String(50)), Column('isopen', Integer, ColumnDefault(1)),
class Order(object): pass
mapper(Order, orders)
The way we will do mapping
class Order(Base): __table__ = orders
Querying
session = Session()q = session.query(Order)print q
.all
.get
.first, .one
Query modification
q = session.query(Order)
.filter
.filter_by
.order_by [desc, asc]
.limit, .offset
Operators
== >= <= > <
~ | ¬_ or_ and_
in_
between like startswith endswith
Some examples
q = session.query(Order)
q.filter_by(user_id=7).order_by(Order.isopen).first()q.filter(Order.description.like('order%')).all()q.filter((Order.user_id==7) | (Order.isopen==1)).all()q.filter(or_(Order.user_id==7, Order.isopen==1)).all()
Slicing: limit/offset sugar
q = session.query(Order)q.limit(1).offset(2)
q[2:3]
Questions
Exercise
● Map the orderitems table to an OrderItem class
● Get a list of all OrderItems– Where they belong to order #3– ... or the item name is “item 1”– ... ordered by item name
(Now would be a good time to look at tutorial_samples.py)
clear_mappers()
Creating, updatingo = Order()o.user_id = 7o.description = 'order 6'session.add(o)o.order_id is Nonesession.commit()o.order_id == 6o.description = 'order B'session.commit()
session.delete(o)session.commit()
Scoped (“smart”) sessions
Session = scoped_session( sessionmaker(autoflush=True, autocommit=False))
assert Session() == Session()
Scoped sessions 2
Base = declarative_base(metadata=Session.metadata)class Order(Base): ...
o = Order()o.user_id = 7o.description = 'order 6'session.commit()
Direct updates, deletes
orders.update(orders.c.order_id==2).execute(isopen=1)orders.delete(orders.c.order_id==2).execute()
SQL layer alert!
One-to-many relations
class User(Base): orders = relation(Order, order_by=[Order.order_id])u = session.query(User).first()print u.orders
Editing collectionso = Order(description='An order')u.orders.append(o)session.commit()
Why sessions are your friends
Some ORMs rely on explicit save
More convenient, less error-prone to let ORM track dirty objects
u = User.get(1)u.orders[0].description = 'An order'u.save() # not real SA code# doh! orders[0] was not saved!
Identity map
Rows with same PK get mapped to same object (per-session)Limited caching for get()Only for get()
Managing the identity map
sesion.query(cls).populate_existing()session.expire(obj)session.refresh(obj)session.expunge(obj)expunge_all, expire_all
Questions
Exercise
Load the user named 'jack' (lowercase)Remove his first orderSave changes to the db
Fun with collectionsu = session.query(User).filter_by(name='jack').one()u.orders = u.orders[1:]session.commit()
>>> session.query(Order).get(1)Order(order_id=1,user_id=None,...)
Two solutions
o = u.orders[0]>>> oOrder(order_id=1,user_id=7,description=u'order 1',isopen=0)session.delete(o)session.commit()
>>> u.orders[Order(order_id=3,...), Order(order_id=5,...)]
Why does it make sense for this to work differently than the previous example?
#2: delete-orphanclass User(Base): __table__ = users orders = relation(Order, cascade="all, delete-orphan", order_by=[Order.order_id])
u = session.query(User).get(7)u.orders = u.orders[1:]session.commit()
>>> session.query(Order).get(1) is NoneTrue
Questions
Exercise
def user_for_order(order): session = Session.object_session(order) return ?
Backrefs
class User(Base): __table__ = users orders = relation(Order, backref='user', order_by=[orders.c.order_id])
o = session.query(Order).first()o.user
That's it for fundamentals
Multi-object queriesOne-to-oneMany-to-manyRelation queriesEager/lazy loadingTransactionsExtensions and related projects
Exercise
List all users who have open orders (isopen is nonzero)
A taste of advanced querying
q = session.query(User).join(User.orders)q.filter(Order.isopen==1).all()
All orders with an open order:
Selecting multiple classes
q = session.query(User, Order) # cartesian join!q = q.join(User.orders)# or!q = q.filter(User.user_id==Order.user_id)
u, o = q.filter(Order.isopen==1).first()
What if we want the user and the order? (efficiently)
Dropping down to SQL
sql = """select u.*from users uwhere u.user_id in ( select user_id from orders where isopen = 1)"""session.query(User).from_statement(sql)
sql = """select u.*, o.*from users u join orders o on (u.user_id = o.user_id)where o.isopen = 1"""session.query(User, Order).from_statement(sql)
Dropping down a little less
q = session.query(User, Order)q = q.join(User.orders)q.filter("orders.isopen = 1").all()
Exercise
In a single query, select the users and orders where the order description is like 'order%'
One to one
class Address(Base): __table__ = addresses
class User(Base): orders = relation(Order, order_by=[orders.c.order_id]) address = relation(Address)
How does SQLA know to treat these differently?
Many to many
class Keyword(Base): __table__ = keywords
class Item(Base): __table__ = orderitems keywords = relation(Keyword, secondary=itemkeywords)
Relation queries
user = session.query(User).get(7)q = session.query(Order)q.filter(Order.user==user).all()# q.filter(Order.user_id==user.user_id).all()
Relation queries 2
q = session.query(Order)
q.filter(Order.user.has(name='jack')).all()q.filter(Order.user.has((User.name=='jack') | (User.user_id >= 9))).all()# q.filter(Order.user_id== select([users.c.user_id], users.c.name=='jack')).all()# q.filter(Order.user_id== select([users.c.user_id], (users.c.name=='jack') | (users.c.user_id >= 9))).all()
Relation queries 3
q = session.query(User)
q.filter(User.orders.any(Order.isopen > 0)).all()q.filter(User.orders.any( Order.description.like('order%'))).all()# q.filter(User.user_id.in_( select([orders.c.user_id], orders.c.isopen > 0))).all()# q.filter(User.user_id.in_( select([orders.c.user_id], orders.c.description.like('order%')))).all()
Relation queries 4
Just need a raw EXISTS clause? Use .any() or .has() without extra parameters.
Relation queries 5
Keyword = session.query(Keyword).filter_by(name='red').one()q = session.query(Item)
q.filter(Item.keywords.contains(keyword)).all()# q.filter(Item.item_id.in_( select([itemkeywords.c.item_id], itemkeywords.c.keyword_id ==keyword.keyword_id))).all()
Exercise
Retrieve all users that do not have any orders.
Breathe Easy
That was our last exercise!
Eager loading
class User(Base): orders = relation(Order, order_by=[orders.c.order_id], lazy=False)# orq = session.query(User).options(eagerload('orders'))
# also lazyload, noload
Transactions are simple
● session: autocommit=False– this is the default– commit() / rollback() manually
● autocommit=True– each flush() also commits
Other transaction features
● begin_nested● manual transaction management at the
Connection level
__init__
class Order(Base): __table__ = orders def __init__(self): self.foo = []
@reconstructor def loaded(self): self.foo = []
__init__ is for object creation, not loadinguse @reconstructor
Questions
Related projects
MigrateFormAlchemySqlSoupElixirz3c.sqlalchemy
Migrate
# one-time setup
migrate create path/to/upgradescripts "comment"
migrate manage dbmanage.py --repository=path/to/upgradescripts –url=db-connection-url
./dbmanage.py version_control
# repeat as necessary:
./dbmanage.py script_sql sqlite# edit script
./dbmanage.py upgrade
FormAlchemy
order1 = session.query(Order).first()
from formalchemy import FieldSet fs = FieldSet(order1) print fs.render()
FormAlchemy, cont.
from formalchemy import Grid orders = session.query(Order).all() g = Grid(Order, orders) print g.render()
SqlSoup
>>> from sqlalchemy.ext.sqlsoup import SqlSoup>>> db = SqlSoup(metadata)
>>> db.users.filter(db.users.user_id < 10).all()[MappedUsers(user_id=7,name='jack'), MappedUsers(user_id=8,name='ed'), MappedUsers(user_id=9,name='fred')]
>>> db.users.first()MappedUsers(user_id=7,name='jack')>>> _.ordersTraceback (most recent call last): File "<stdin>", line 1, in <module>AttributeError: 'MappedUsers' object has no attribute 'orders'
SqlSoup 2
>>> db.users.relate('orders', db.orders)>>> db.users.first().orders[MappedOrders(...)]
>>> db.users.filter(db.users.orders.any()).all()[MappedUsers(...)]
SqlSoup 3
s = select([func.count('*')], users.c.user_id==orders.c.user_id, from_obj=[orders], scalar=True)s2 = select([users, s.label('order_count')]).alias('users_with_count')db.users_with_count = db.map(s2)
Elixir
class Person(Entity): has_field('name', Unicode) acts_as_taggable() ... some_person_instance.add_tag('cool') ... cool_people = Person.get_by_tag('cool')
http://cleverdevil.org/computing/52/
Resources
irc://irc.freenode.net/#sqlalchemySQLA mailing listMigrate, FormAlchemy, Elixir
Final Questions