SQL Alchemy 0.7.3 manual

SQLAlchemy DocumentationRelease 0.7.3

Mike Bayer

October 17, 2011

CONTENTS

1 Overview 31.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.2 Documentation Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.3 Code Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.4 Installation Guide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

1.4.1 Supported Platforms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.4.2 Supported Installation Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.4.3 Install via easy_install or pip . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.4.4 Installing using setup.py . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.4.5 Installing the C Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.4.6 Installing on Python 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.4.7 Installing a Database API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.4.8 Checking the Installed SQLAlchemy Version . . . . . . . . . . . . . . . . . . . . . . . . . 5

1.5 0.6 to 0.7 Migration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2 SQLAlchemy ORM 72.1 Object Relational Tutorial . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2.1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72.1.2 Version Check . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72.1.3 Connecting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72.1.4 Declare a Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82.1.5 Create an Instance of the Mapped Class . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102.1.6 Creating a Session . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112.1.7 Adding New Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122.1.8 Rolling Back . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142.1.9 Querying . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

Common Filter Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17Returning Lists and Scalars . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18Using Literal SQL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19Counting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

2.1.10 Building a Relationship . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222.1.11 Working with Related Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242.1.12 Querying with Joins . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

Using Aliases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26Using Subqueries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27Selecting Entities from Subqueries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27Using EXISTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28Common Relationship Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

2.1.13 Eager Loading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

i

Subquery Load . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30Joined Load . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30Explicit Join + Eagerload . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

2.1.14 Deleting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32Configuring delete/delete-orphan Cascade . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

2.1.15 Building a Many To Many Relationship . . . . . . . . . . . . . . . . . . . . . . . . . . . . 352.1.16 Further Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

2.2 Mapper Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 382.2.1 Classical Mappings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 392.2.2 Customizing Column Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

Naming Columns Distinctly from Attribute Names . . . . . . . . . . . . . . . . . . . . . . . 40Mapping Multiple Columns to a Single Attribute . . . . . . . . . . . . . . . . . . . . . . . . 41Using column_property for column level options . . . . . . . . . . . . . . . . . . . . . . . . 42Mapping a Subset of Table Columns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

2.2.3 Deferred Column Loading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44Column Deferral API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

2.2.4 SQL Expressions as Mapped Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46Alternatives to column_property() . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

2.2.5 Changing Attribute Behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48Simple Validators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48Using Descriptors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49Synonyms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51Custom Comparators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

2.2.6 Composite Column Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54Tracking In-Place Mutations on Composites . . . . . . . . . . . . . . . . . . . . . . . . . . . 56Redefining Comparison Operations for Composites . . . . . . . . . . . . . . . . . . . . . . . 56

2.2.7 Mapping a Class against Multiple Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . 562.2.8 Mapping a Class against Arbitrary Selects . . . . . . . . . . . . . . . . . . . . . . . . . . . 572.2.9 Multiple Mappers for One Class . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 582.2.10 Multiple Persistence Mappers for One Class . . . . . . . . . . . . . . . . . . . . . . . . . 582.2.11 Constructors and Object Initialization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 592.2.12 Class Mapping API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

2.3 Relationship Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 672.3.1 Basic Relational Patterns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

One To Many . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68Many To One . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68One To One . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69Many To Many . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69Association Object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

2.3.2 Adjacency List Relationships . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71Self-Referential Query Strategies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72Configuring Self-Referential Eager Loading . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

2.3.3 Linking Relationships with Backref . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74Backref Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76One Way Backrefs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

2.3.4 Specifying Alternate Join Conditions to relationship() . . . . . . . . . . . . . . . . . . . . . 78Self-Referential Many-to-Many Relationship . . . . . . . . . . . . . . . . . . . . . . . . . . 80Specifying Foreign Keys . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81Building Query-Enabled Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81Multiple Relationships against the Same Parent/Child . . . . . . . . . . . . . . . . . . . . . . 82

2.3.5 Rows that point to themselves / Mutually Dependent Rows . . . . . . . . . . . . . . . . . . 822.3.6 Mutable Primary Keys / Update Cascades . . . . . . . . . . . . . . . . . . . . . . . . . . . 832.3.7 Relationships API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

2.4 Collection Configuration and Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

ii

2.4.1 Working with Large Collections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90Dynamic Relationship Loaders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90Setting Noload . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91Using Passive Deletes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

2.4.2 Customizing Collection Access . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91Dictionary Collections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92Custom Collection Implementations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94Collections API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

2.5 Mapping Class Inheritance Hierarchies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1012.5.1 Joined Table Inheritance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102

Basic Control of Which Tables are Queried . . . . . . . . . . . . . . . . . . . . . . . . . . . 103Advanced Control of Which Tables are Queried . . . . . . . . . . . . . . . . . . . . . . . . . 105Creating Joins to Specific Subtypes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105

2.5.2 Single Table Inheritance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1072.5.3 Concrete Table Inheritance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107

Concrete Inheritance with Declarative . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1092.5.4 Using Relationships with Inheritance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109

Relationships with Concrete Inheritance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1092.5.5 Using Inheritance with Declarative . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111

2.6 Using the Session . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1112.6.1 What does the Session do ? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1112.6.2 Getting a Session . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111

Adding Additional Configuration to an Existing sessionmaker() . . . . . . . . . . . . . . . . 112Creating Ad-Hoc Session Objects with Alternate Arguments . . . . . . . . . . . . . . . . . . 112

2.6.3 Using the Session . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113Quickie Intro to Object States . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113Session Frequently Asked Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113Querying . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115Adding New or Existing Items . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115Merging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116Deleting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118Flushing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119Committing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120Rolling Back . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120Expunging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121Closing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121Refreshing / Expiring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121Session Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122

2.6.4 Cascades . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1222.6.5 Managing Transactions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124

Using SAVEPOINT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125Autocommit Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125Enabling Two-Phase Commit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127

2.6.6 Embedding SQL Insert/Update Expressions into a Flush . . . . . . . . . . . . . . . . . . . 1272.6.7 Using SQL Expressions with Sessions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1282.6.8 Joining a Session into an External Transaction . . . . . . . . . . . . . . . . . . . . . . . . . 1282.6.9 Contextual/Thread-local Sessions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129

Creating a Thread-local Context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129Lifespan of a Contextual Session . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130Contextual Session API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131

2.6.10 Partitioning Strategies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132Vertical Partitioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132Horizontal Partitioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133

2.6.11 Sessions API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133

iii

Session and sessionmaker() . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133Session Utilites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142Attribute and State Management Utilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142

2.7 Querying . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1442.7.1 The Query Object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1442.7.2 ORM-Specific Query Constructs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158

2.8 Relationship Loading Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1602.8.1 Using Loader Strategies: Lazy Loading, Eager Loading . . . . . . . . . . . . . . . . . . . . 1602.8.2 The Zen of Eager Loading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1622.8.3 What Kind of Loading to Use ? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1642.8.4 Routing Explicit Joins/Statements into Eagerly Loaded Collections . . . . . . . . . . . . . . 1652.8.5 Relation Loader API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166

2.9 ORM Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1692.9.1 Attribute Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1692.9.2 Mapper Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1712.9.3 Instance Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1772.9.4 Session Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1792.9.5 Instrumentation Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1812.9.6 Alternate Class Instrumentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182

2.10 ORM Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1822.10.1 Association Proxy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183

Simplifying Scalar Collections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183Creation of New Values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184Simplifying Association Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185Proxying to Dictionary Based Collections . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186Composite Association Proxies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188Querying with Association Proxies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189API Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190

2.10.2 Declarative . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193Synopsis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193Defining Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193Accessing the MetaData . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194Configuring Relationships . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194Configuring Many-to-Many Relationships . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195Defining SQL Expressions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195Table Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195Using a Hybrid Approach with __table__ . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196Mapper Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197Inheritance Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197Mixin and Custom Base Classes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200Special Directives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206Class Constructor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207Sessions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207API Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207

2.10.3 Mutation Tracking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210Establishing Mutability on Scalar Column Values . . . . . . . . . . . . . . . . . . . . . . . . 210Establishing Mutability on Composites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213API Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215

2.10.4 Ordering List . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216API Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218

2.10.5 Horizontal Sharding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218API Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218

2.10.6 Hybrid Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219Defining Expression Behavior Distinct from Attribute Behavior . . . . . . . . . . . . . . . . 220

iv

Defining Setters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221Working with Relationships . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222Building Custom Comparators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223Hybrid Value Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224API Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225

2.10.7 SqlSoup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226Loading objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227Modifying objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228Joins . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228Relationships . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229Advanced Use . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230SqlSoup API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231

2.11 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2342.11.1 Adjacency List . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2342.11.2 Associations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2342.11.3 Attribute Instrumentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2342.11.4 Beaker Caching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2352.11.5 Directed Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2362.11.6 Dynamic Relations as Dictionaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2362.11.7 Generic Associations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2362.11.8 Horizontal Sharding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2362.11.9 Inheritance Mappings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2372.11.10 Large Collections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2372.11.11 Nested Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2372.11.12 Polymorphic Associations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2372.11.13 PostGIS Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2372.11.14 Versioned Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2382.11.15 Vertical Attribute Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2392.11.16 XML Persistence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239

2.12 Deprecated ORM Event Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2402.12.1 Mapper Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2402.12.2 Session Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2432.12.3 Attribute Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245

2.13 ORM Exceptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2462.14 ORM Internals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247

3 SQLAlchemy Core 2553.1 SQL Expression Language Tutorial . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255

3.1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2553.1.2 Version Check . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2553.1.3 Connecting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2563.1.4 Define and Create Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2563.1.5 Insert Expressions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2573.1.6 Executing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2583.1.7 Executing Multiple Statements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2583.1.8 Connectionless / Implicit Execution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2593.1.9 Selecting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2603.1.10 Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2623.1.11 Conjunctions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2633.1.12 Using Text . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2643.1.13 Using Aliases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2653.1.14 Using Joins . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2663.1.15 Intro to Generative Selects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267

v

Transforming a Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2683.1.16 Everything Else . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268

Bind Parameter Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269Window Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271Unions and Other Set Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271Scalar Selects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272Correlated Subqueries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272Ordering, Grouping, Limiting, Offset...ing... . . . . . . . . . . . . . . . . . . . . . . . . . . . 273

3.1.17 Inserts and Updates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273Correlated Updates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275

3.1.18 Deletes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2753.1.19 Further Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275

3.2 SQL Statements and Expressions API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2763.2.1 Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2763.2.2 Classes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2893.2.3 Generic Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 310

3.3 Engine Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3123.3.1 Supported Databases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3123.3.2 Engine Creation API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3143.3.3 Database Urls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3163.3.4 Custom DBAPI connect() arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3183.3.5 Configuring Logging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 318

3.4 Working with Engines and Connections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3193.4.1 Basic Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3193.4.2 Using Transactions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 320

Nesting of Transaction Blocks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3213.4.3 Understanding Autocommit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3213.4.4 Connectionless Execution, Implicit Execution . . . . . . . . . . . . . . . . . . . . . . . . . 3223.4.5 Using the Threadlocal Execution Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . 3233.4.6 Connection / Engine API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 324

3.5 Connection Pooling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3353.5.1 Connection Pool Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3353.5.2 Switching Pool Implementations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3363.5.3 Using a Custom Connection Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3363.5.4 Constructing a Pool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3363.5.5 Pool Event Listeners . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3373.5.6 Dealing with Disconnects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 337

Disconnect Handling - Optimistic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 337Disconnect Handling - Pessimistic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 338

3.5.7 API Documentation - Available Pool Implementations . . . . . . . . . . . . . . . . . . . . 3393.5.8 Pooling Plain DB-API Connections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 342

3.6 Schema Definition Language . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3433.6.1 Describing Databases with MetaData . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343

Accessing Tables and Columns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343Creating and Dropping Database Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 345Binding MetaData to an Engine or Connection . . . . . . . . . . . . . . . . . . . . . . . . . 346Specifying the Schema Name . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 347Backend-Specific Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 348Column, Table, MetaData API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 348

3.6.2 Reflecting Database Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 356Overriding Reflected Columns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 357Reflecting Views . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 357Reflecting All Tables at Once . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 357

vi

Fine Grained Reflection with Inspector . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3573.6.3 Column Insert/Update Defaults . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 360

Scalar Defaults . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 360Python-Executed Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361SQL Expressions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 362Server Side Defaults . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 363Triggered Columns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 363Defining Sequences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 363Default Objects API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 364

3.6.4 Defining Constraints and Indexes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 366Defining Foreign Keys . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 366UNIQUE Constraint . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 369CHECK Constraint . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 369Setting up Constraints when using the Declarative ORM Extension . . . . . . . . . . . . . . 370Constraints API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 370Indexes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373

3.6.5 Customizing DDL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375Controlling DDL Sequences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375Custom DDL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 377DDL Expression Constructs API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 378

3.7 Column and Data Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3823.7.1 Generic Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3823.7.2 SQL Standard Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3893.7.3 Vendor-Specific Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3913.7.4 Custom Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 391

Overriding Type Compilation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 391Augmenting Existing Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 392TypeDecorator Recipes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 396Creating New Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 398

3.7.5 Base Type API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4003.8 Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 404

3.8.1 Event Registration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4043.8.2 Targets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4043.8.3 Modifiers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4053.8.4 Event Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4053.8.5 API Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 405

3.9 Core Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4063.9.1 Connection Pool Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4063.9.2 Connection Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4073.9.3 Schema Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 408

3.10 Custom SQL Constructs and Compilation Extension . . . . . . . . . . . . . . . . . . . . . . . . . . 4113.10.1 Synopsis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4113.10.2 Dialect-specific compilation rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4123.10.3 Compiling sub-elements of a custom expression construct . . . . . . . . . . . . . . . . . . 412

Cross Compiling between SQL and DDL compilers . . . . . . . . . . . . . . . . . . . . . . . 4133.10.4 Enabling Autocommit on a Construct . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4133.10.5 Changing the default compilation of existing constructs . . . . . . . . . . . . . . . . . . . . 4143.10.6 Changing Compilation of Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4143.10.7 Subclassing Guidelines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4143.10.8 Further Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 415

UTC timestamp function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 415GREATEST function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 416false expression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 417

3.11 Expression Serializer Extension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 417

vii

3.12 Deprecated Event Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4183.12.1 Execution, Connection and Cursor Events . . . . . . . . . . . . . . . . . . . . . . . . . . . 4183.12.2 Connection Pool Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 419

3.13 Core Exceptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4203.14 Core Internals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 424

4 Dialects 4354.1 Drizzle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 435

4.1.1 Supported Versions and Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4354.1.2 Connecting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4354.1.3 Connection Timeouts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4354.1.4 Storage Engines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4364.1.5 Keys . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4364.1.6 Drizzle SQL Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4364.1.7 Drizzle Data Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4374.1.8 MySQL-Python Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 440

Connecting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 440Character Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 440Known Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 440

4.2 Firebird . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4404.2.1 Dialects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4414.2.2 Locking Behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4414.2.3 RETURNING support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4414.2.4 kinterbasdb . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 441

4.3 Informix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4424.3.1 informixdb Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 442

Connecting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4424.4 MaxDB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 442

4.4.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4424.4.2 Connecting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4434.4.3 Implementation Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 443

sapdb.dbapi . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4444.5 Microsoft Access . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4444.6 Microsoft SQL Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 444

4.6.1 Connecting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4444.6.2 Auto Increment Behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4444.6.3 Collation Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4454.6.4 LIMIT/OFFSET Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4454.6.5 Nullability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4454.6.6 Date / Time Handling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4464.6.7 Compatibility Levels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4464.6.8 Triggers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4464.6.9 Enabling Snapshot Isolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4464.6.10 Scalar Select Comparisons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4474.6.11 Known Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4474.6.12 SQL Server Data Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4474.6.13 PyODBC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 450

Connecting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4504.6.14 mxODBC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 451

Connecting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 451Execution Modes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 451

4.6.15 pymssql . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 451Connecting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 452Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 452

viii

4.6.16 zxjdbc Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 452JDBC Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 452Connecting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 452

4.6.17 AdoDBAPI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4524.7 MySQL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 452

4.7.1 Supported Versions and Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4534.7.2 Connecting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4534.7.3 Connection Timeouts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4534.7.4 Storage Engines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4534.7.5 Case Sensitivity and Table Reflection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4534.7.6 Keys . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4544.7.7 SQL Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4544.7.8 MySQL SQL Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4554.7.9 CAST Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4554.7.10 MySQL Specific Index Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 455

Index Length . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4554.7.11 MySQL Data Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4564.7.12 MySQL-Python Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 465

Connecting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 465Character Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 465Known Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 465

4.7.13 OurSQL Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 466Connecting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 466Character Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 466

4.7.14 pymysql Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 466Connecting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 466MySQL-Python Compatibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 466

4.7.15 MySQL-Connector Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 467Connecting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 467

4.7.16 pyodbc Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 467Connecting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 467Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 467

4.7.17 zxjdbc Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 467JDBC Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 467Connecting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 467Character Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 468

4.8 Oracle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4684.8.1 Connect Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4684.8.2 Auto Increment Behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4684.8.3 Identifier Casing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4684.8.4 Unicode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4694.8.5 LIMIT/OFFSET Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4694.8.6 ON UPDATE CASCADE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4694.8.7 Oracle 8 Compatibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4694.8.8 Synonym/DBLINK Reflection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4704.8.9 Oracle Data Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4704.8.10 cx_Oracle Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 472

Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 473Connecting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 473Unicode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 473LOB Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 473Two Phase Transaction Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 473Precision Numerics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 474

4.8.11 zxjdbc Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 474

ix

JDBC Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4744.9 PostgreSQL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 474

4.9.1 Sequences/SERIAL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4744.9.2 Transaction Isolation Level . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4754.9.3 Remote / Cross-Schema Table Introspection . . . . . . . . . . . . . . . . . . . . . . . . . . 4754.9.4 INSERT/UPDATE...RETURNING . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4754.9.5 Postgresql-Specific Index Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 476

Partial Indexes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 476Operator Classes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 476Index Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 476

4.9.6 PostgreSQL Data Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4764.9.7 psycopg2 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 479

Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 480Connecting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 480Per-Statement/Connection Execution Options . . . . . . . . . . . . . . . . . . . . . . . . . . 480Unicode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 480Transactions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 480Client Encoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 481Transaction Isolation Level . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 481NOTICE logging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 481

4.9.8 py-postgresql Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 481Connecting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 481

4.9.9 pg8000 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 481Connecting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 481Unicode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 481Interval . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 482

4.9.10 zxjdbc Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 482JDBC Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 482

4.10 SQLite . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4824.10.1 Date and Time Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4824.10.2 Auto Incrementing Behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4824.10.3 Transaction Isolation Level . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4834.10.4 SQLite Data Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4834.10.5 Pysqlite . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 484

Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 485Connect Strings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 485Compatibility with sqlite3 native date and datetime types . . . . . . . . . . . . . . . . . . . 485Threading/Pooling Behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 486Unicode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 487

4.11 Sybase . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4874.11.1 python-sybase notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 487

Unicode Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4874.11.2 pyodbc notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 487

Unicode Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4884.11.3 mxodbc notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 488

5 Indices and tables 489

Python Module Index 491

Index 493

x

SQLAlchemy Documentation, Release 0.7.3

Full table of contents. For a high level overview of all documentation, see index_toplevel.

CONTENTS 1


2 CONTENTS

CHAPTER

ONE

OVERVIEW

1.1 Overview

The SQLAlchemy SQL Toolkit and Object Relational Mapper is a comprehensive set of tools for working withdatabases and Python. It has several distinct areas of functionality which can be used individually or combinedtogether. Its major components are illustrated in below, with component dependencies organized into layers:

Above, the two most significant front-facing portions of SQLAlchemy are the Object Relational Mapper and theSQL Expression Language. SQL Expressions can be used independently of the ORM. When using the ORM, theSQL Expression language remains part of the public facing API as it is used within object-relational configurationsand queries.

3


1.2 Documentation Overview

The documentation is separated into three sections: SQLAlchemy ORM, SQLAlchemy Core, and Dialects.

In SQLAlchemy ORM, the Object Relational Mapper is introduced and fully described. New users should begin withthe Object Relational Tutorial. If you want to work with higher-level SQL which is constructed automatically for you,as well as management of Python objects, proceed to this tutorial.

In SQLAlchemy Core, the breadth of SQLAlchemys SQL and database integration and description services are doc-umented, the core of which is the SQL Expression language. The SQL Expression Language is a toolkit all itsown, independent of the ORM package, which can be used to construct manipulable SQL expressions which can beprogrammatically constructed, modified, and executed, returning cursor-like result sets. In contrast to the ORMsdomain-centric mode of usage, the expression language provides a schema-centric usage paradigm. New users shouldbegin here with SQL Expression Language Tutorial. SQLAlchemy engine, connection, and pooling services are alsodescribed in SQLAlchemy Core.

In Dialects, reference documentation for all provided database and DBAPI backends is provided.

1.3 Code Examples

Working code examples, mostly regarding the ORM, are included in the SQLAlchemy distribution. A description ofall the included example applications is at Examples.

There is also a wide variety of examples involving both core SQLAlchemy constructs as well as the ORM on the wiki.See Theatrum Chemicum.

1.4 Installation Guide

1.4.1 Supported Platforms

SQLAlchemy has been tested against the following platforms:

cPython since version 2.4, through the 2.xx series

cPython version 3, throughout all 3.xx series

Jython 2.5 or greater

Pypy 1.5 or greater

1.4.2 Supported Installation Methods

SQLAlchemy supports installation using standard Python distutils or setuptools methodologies. An overview ofpotential setups is as follows:

Plain Python Distutils - SQLAlchemy can be installed with a clean Python install using the services providedvia Python Distutils, using the setup.py script. The C extensions as well as Python 3 builds are supported.

Standard Setuptools - When using setuptools, SQLAlchemy can be installed via setup.py oreasy_install, and the C extensions are supported. setuptools is not supported on Python 3 at the timeof of this writing.

Distribute - With distribute, SQLAlchemy can be installed via setup.py or easy_install, and the Cextensions as well as Python 3 builds are supported.

4 Chapter 1. Overview


pip - pip is an installer that rides on top of setuptools or distribute, replacing the usage ofeasy_install. It is often preferred for its simpler mode of usage.

1.4.3 Install via easy_install or pip

When easy_install or pip is available, the distribution can be downloaded from Pypi and installed in one step:

easy_install SQLAlchemy

Or with pip:

pip install SQLAlchemy

This command will download the latest version of SQLAlchemy from the Python Cheese Shop and install it to yoursystem.

1.4.4 Installing using setup.py

Otherwise, you can install from the distribution using the setup.py script:

python setup.py install

1.4.5 Installing the C Extensions

SQLAlchemy includes C extensions which provide an extra speed boost for dealing with result sets. Currently, theextensions are only supported on the 2.xx series of cPython, not Python 3 or Pypy.

setup.py will automatically build the extensions if an appropriate platform is detected. If the build of the C extensionsfails, due to missing compiler or other issue, the setup process will output a warning message, and re-run the buildwithout the C extensions, upon completion reporting final status.

To run the build/install without even attempting to compile the C extensions, pass the flag--without-cextensions to the setup.py script:

python setup.py --without-cextensions install

1.4.6 Installing on Python 3

SQLAlchemy ships as Python 2 code. For Python 3 usage, the setup.py script will invoke the Python 2to3 tool onthe build, plugging in an extra preprocessor as well. The 2to3 step works with Python distutils (part of the standardPython install) and Distribute - it will not work with a non-Distribute setuptools installation.

1.4.7 Installing a Database API

SQLAlchemy is designed to operate with a DB-API implementation built for a particular database, and includessupport for the most popular databases. The current list is at Supported Databases.

1.4.8 Checking the Installed SQLAlchemy Version

This documentation covers SQLAlchemy version 0.7. If youre working on a system that already has SQLAlchemyinstalled, check the version from your Python prompt like this:

1.4. Installation Guide 5


>>> import sqlalchemy>>> sqlalchemy.__version__0.7.0

1.5 0.6 to 0.7 Migration

Notes on whats changed from 0.6 to 0.7 is available on the SQLAlchemy wiki at 07Migration.

6 Chapter 1. Overview

CHAPTER

TWO

SQLALCHEMY ORM

2.1 Object Relational Tutorial

2.1.1 Introduction

The SQLAlchemy Object Relational Mapper presents a method of associating user-defined Python classes withdatabase tables, and instances of those classes (objects) with rows in their corresponding tables. It includes a sys-tem that transparently synchronizes all changes in state between objects and their related rows, called a unit of work,as well as a system for expressing database queries in terms of the user defined classes and their defined relationshipsbetween each other.

The ORM is in contrast to the SQLAlchemy Expression Language, upon which the ORM is constructed. Whereas theSQL Expression Language, introduced in SQL Expression Language Tutorial, presents a system of representing theprimitive constructs of the relational database directly without opinion, the ORM presents a high level and abstractedpattern of usage, which itself is an example of applied usage of the Expression Language.

While there is overlap among the usage patterns of the ORM and the Expression Language, the similarities are moresuperficial than they may at first appear. One approaches the structure and content of data from the perspective ofa user-defined domain model which is transparently persisted and refreshed from its underlying storage model. Theother approaches it from the perspective of literal schema and SQL expression representations which are explicitlycomposed into messages consumed individually by the database.

A successful application may be constructed using the Object Relational Mapper exclusively. In advanced situations,an application constructed with the ORM may make occasional usage of the Expression Language directly in certainareas where specific database interactions are required.

The following tutorial is in doctest format, meaning each >>> line represents something you can type at a Pythoncommand prompt, and the following text represents the expected return value.

2.1.2 Version Check

A quick check to verify that we are on at least version 0.7 of SQLAlchemy:

>>> import sqlalchemy>>> sqlalchemy.__version__0.7.0

2.1.3 Connecting

For this tutorial we will use an in-memory-only SQLite database. To connect we use create_engine():

7


>>> from sqlalchemy import create_engine>>> engine = create_engine(sqlite:///:memory:, echo=True)

The echo flag is a shortcut to setting up SQLAlchemy logging, which is accomplished via Pythons standardlogging module. With it enabled, well see all the generated SQL produced. If you are working through thistutorial and want less output generated, set it to False. This tutorial will format the SQL behind a popup window soit doesnt get in our way; just click the SQL links to see whats being generated.

The return value of create_engine() is an instance of Engine, and it represents the core interface to thedatabase, adapted through a dialect that handles the details of the database and DBAPI in use. In this case the SQLitedialect will interpret instructions to the Python built-in sqlite3 module.

The Engine has not actually tried to connect to the database yet; that happens only the first time it is asked to performa task against the database. We can illustrate this by asking it to perform a simple SELECT statement:

>>> engine.execute("select 1").scalar()select 1()1

As the Engine.execute() method is called, the Engine establishes a connection to the SQLite database, whichis then used to emit the SQL. The connection is then returned to an internal connection pool where it will be reused onsubsequent statement executions. While we illustrate direct usage of the Engine here, this isnt typically necessarywhen using the ORM, where the Engine, once created, is used behind the scenes by the ORM as well see shortly.

2.1.4 Declare a Mapping

When using the ORM, the configurational process starts by describing the database tables well be dealing with, andthen by defining our own classes which will be mapped to those tables. In modern SQLAlchemy, these two tasksare usually performed together, using a system known as Declarative, which allows us to create classes that includedirectives to describe the actual database table they will be mapped to.

Classes mapped using the Declarative system are defined in terms of a base class which maintains a catalog of classesand tables relative to that base - this is known as the declarative base class. Our application will usually have just oneinstance of this base in a commonly imported module. We create the base class using the declarative_base()function, as follows:

>>> from sqlalchemy.ext.declarative import declarative_base

>>> Base = declarative_base()

Now that we have a base, we can define any number of mapped classes in terms of it. We will start with just a singletable called users, which will store records for the end-users using our application. A new class called User willbe the class to which we map this table. The imports well need to accomplish this include objects that represent thecomponents of our table, including the Column class which represents a database column, as well as the Integerand String classes that represent basic datatypes used in columns:

>>> from sqlalchemy import Column, Integer, String>>> class User(Base):... __tablename__ = users...... id = Column(Integer, primary_key=True)... name = Column(String)... fullname = Column(String)... password = Column(String)...... def __init__(self, name, fullname, password):... self.name = name

8 Chapter 2. SQLAlchemy ORM


... self.fullname = fullname

... self.password = password

...

... def __repr__(self):

... return "" % (self.name, self.fullname, self.password)

The above User class establishes details about the table being mapped, including the name of the table denoted bythe __tablename__ attribute, a set of columns id, name, fullname and password, where the id column willalso be the primary key of the table. While its certainly possible that some database tables dont have primary keycolumns (as is also the case with views, which can also be mapped), the ORM in order to actually map to a particulartable needs there to be at least one column denoted as a primary key column; multiple-column, i.e. composite, primarykeys are of course entirely feasible as well.

We define a constructor via __init__() and also a __repr__() method - both are optional. The class of coursecan have any number of other methods and attributes as required by the application, as its basically just a plainPython class. Inheriting from Base is also only a requirement of the declarative configurational system, which itselfis optional and relatively open ended; at its core, the SQLAlchemy ORM only requires that a class be a so-called newstyle class, that is, it inherits from object in Python 2, in order to be mapped. All classes in Python 3 are newstyle classes.

The Non Opinionated Philosophy

In our User mapping example, it was required that we identify the name of the table in use, as well as thenames and characteristics of all columns which we care about, including which column or columns represent theprimary key, as well as some basic information about the types in use. SQLAlchemy never makes assumptionsabout these decisions - the developer must always be explicit about specific conventions in use. However, thatdoesnt mean the task cant be automated. While this tutorial will keep things explicit, developers are encouragedto make use of helper functions as well as Declarative Mixins to automate their tasks in large scale applications.The section Mixin and Custom Base Classes introduces many of these techniques.

With our User class constructed via the Declarative system, we have defined information about our table, known astable metadata, as well as a user-defined class which is linked to this table, known as a mapped class. Declarative hasprovided for us a shorthand system for what in SQLAlchemy is called a Classical Mapping, which specifies thesetwo units separately and is discussed in Classical Mappings. The table is actually represented by a datastructure knownas Table, and the mapping represented by a mapper object generated by a function called mapper(). Declarativeperforms both of these steps for us, making available the Table it has created via the __table__ attribute:

>>> User.__table__Table(users, MetaData(None),

Column(id, Integer(), table=, primary_key=True, nullable=False),Column(name, String(), table=),Column(fullname, String(), table=),Column(password, String(), table=), schema=None)

and while rarely needed, making available the mapper() object via the __mapper__ attribute:

>>> User.__mapper__

The Declarative base class also contains a catalog of all the Table objects that have been defined called MetaData,available via the .metadata attribute. In this example, we are defining new tables that have yet to be created in ourSQLite database, so one helpful feature the MetaData object offers is the ability to issue CREATE TABLE statementsto the database for all tables that dont yet exist. We illustrate this by calling the MetaData.create_all()method, passing in our Engine as a source of database connectivity. We will see that special commands are firstemitted to check for the presence of the users table, and following that the actual CREATE TABLE statement:

>>> Base.metadata.create_all(engine)

2.1. Object Relational Tutorial 9


PRAGMA table_info("users")()CREATE TABLE users (

id INTEGER NOT NULL,name VARCHAR,fullname VARCHAR,password VARCHAR,PRIMARY KEY (id)

)()COMMIT

Minimal Table Descriptions vs. Full Descriptions

Users familiar with the syntax of CREATE TABLE may notice that the VARCHAR columns were generatedwithout a length; on SQLite and Postgresql, this is a valid datatype, but on others, its not allowed. So if runningthis tutorial on one of those databases, and you wish to use SQLAlchemy to issue CREATE TABLE, a lengthmay be provided to the String type as below:Column(String(50))The length field on String, as well as similar precision/scale fields available on Integer, Numeric, etc.are not referenced by SQLAlchemy other than when creating tables.Additionally, Firebird and Oracle require sequences to generate new primary key identifiers, and SQLAlchemydoesnt generate or assume these without being instructed. For that, you use the Sequence construct:from sqlalchemy import SequenceColumn(Integer, Sequence(user_id_seq), primary_key=True)A full, foolproof Table generated via our declarative mapping is therefore:class User(Base):

__tablename__ = usersid = Column(Integer, Sequence(user_id_seq), primary_key=True)name = Column(String(50))fullname = Column(String(50))password = Column(String(12))

def __init__(self, name, fullname, password):self.name = nameself.fullname = fullnameself.password = password

def __repr__(self):return "" % (self.name, self.fullname, self.password)

We include this more verbose table definition separately to highlight the difference between a minimal constructgeared primarily towards in-Python usage only, versus one that will be used to emit CREATE TABLE statementson a particular set of backends with more stringent requirements.

2.1.5 Create an Instance of the Mapped Class

With mappings complete, lets now create and inspect a User object:

>>> ed_user = User(ed, Ed Jones, edspassword)>>> ed_user.nameed>>> ed_user.password



edspassword>>> str(ed_user.id)None

The id attribute, which while not defined by our __init__() method, exists with a value of None on our Userinstance due to the id column we declared in our mapping. By default, the ORM creates class attributes for all columnspresent in the table being mapped. These class attributes exist as Python descriptors, and define instrumentationfor the mapped class. The functionality of this instrumentation includes the ability to fire on change events, trackmodifications, and to automatically load new data from the database when needed.

Since we have not yet told SQLAlchemy to persist Ed Jones within the database, its id is None. When we persistthe object later, this attribute will be populated with a newly generated value.

The default __init__() method

Note that in our User example we supplied an __init__() method, which receives name, fullname andpassword as positional arguments. The Declarative system supplies for us a default constructor if one is notalready present, which accepts keyword arguments of the same name as that of the mapped attributes. Below wedefine User without specifying a constructor:class User(Base):

__tablename__ = usersid = Column(Integer, primary_key=True)name = Column(String)fullname = Column(String)password = Column(String)

Our User class above will make usage of the default constructor, and provide id, name, fullname, andpassword as keyword arguments:u1 = User(name=ed, fullname=Ed Jones, password=foobar)

2.1.6 Creating a Session

Were now ready to start talking to the database. The ORMs handle to the database is the Session. When we firstset up the application, at the same level as our create_engine() statement, we define a Session class whichwill serve as a factory for new Session objects:

>>> from sqlalchemy.orm import sessionmaker>>> Session = sessionmaker(bind=engine)

In the case where your application does not yet have an Engine when you define your module-level objects, just setit up like this:

>>> Session = sessionmaker()

Later, when you create your engine with create_engine(), connect it to the Session using configure():

>>> Session.configure(bind=engine) # once engine is available

This custom-made Session class will create new Session objects which are bound to our database. Other transac-tional characteristics may be defined when calling sessionmaker() as well; these are described in a later chapter.Then, whenever you need to have a conversation with the database, you instantiate a Session:

>>> session = Session()

The above Session is associated with our SQLite-enabled Engine, but it hasnt opened any connections yet. Whenits first used, it retrieves a connection from a pool of connections maintained by the Engine, and holds onto it untilwe commit all changes and/or close the session object.



Session Creational Patterns

The business of acquiring a Session has a good deal of variety based on the variety of types of applicationsand frameworks out there. Keep in mind the Session is just a workspace for your objects, local to a particulardatabase connection - if you think of an application thread as a guest at a dinner party, the Session is theguests plate and the objects it holds are the food (and the database...the kitchen?)! Hints on how Session isintegrated into an application are at Session Frequently Asked Questions.

2.1.7 Adding New Objects

To persist our User object, we add() it to our Session:

>>> ed_user = User(ed, Ed Jones, edspassword)>>> session.add(ed_user)

At this point, we say that the instance is pending; no SQL has yet been issued and the object is not yet represented bya row in the database. The Session will issue the SQL to persist Ed Jones as soon as is needed, using a processknown as a flush. If we query the database for Ed Jones, all pending information will first be flushed, and the queryis issued immediately thereafter.

For example, below we create a new Query object which loads instances of User. We filter by the name attributeof ed, and indicate that wed like only the first result in the full list of rows. A User instance is returned which isequivalent to that which weve added:

>>> our_user = session.query(User).filter_by(name=ed).first()BEGIN (implicit)INSERT INTO users (name, fullname, password) VALUES (?, ?, ?)(ed, Ed Jones, edspassword)SELECT users.id AS users_id,

users.name AS users_name,users.fullname AS users_fullname,users.password AS users_password

FROM usersWHERE users.name = ?LIMIT ? OFFSET ?

(ed, 1, 0)>>> our_user

In fact, the Session has identified that the row returned is the same row as one already represented within its internalmap of objects, so we actually got back the identical instance as that which we just added:

>>> ed_user is our_userTrue

The ORM concept at work here is known as an identity map and ensures that all operations upon a particular rowwithin a Session operate upon the same set of data. Once an object with a particular primary key is present in theSession, all SQL queries on that Session will always return the same Python object for that particular primarykey; it also will raise an error if an attempt is made to place a second, already-persisted object with the same primarykey within the session.

We can add more User objects at once using add_all():

>>> session.add_all([... User(wendy, Wendy Williams, foobar),



... User(mary, Mary Contrary, xxg527),

... User(fred, Fred Flinstone, blah)])

Also, Ed has already decided his password isnt too secure, so lets change it:

>>> ed_user.password = f8s7ccs

The Session is paying attention. It knows, for example, that Ed Jones has been modified:

>>> session.dirtyIdentitySet([])

and that three new User objects are pending:

>>> session.newIdentitySet([,,])

We tell the Session that wed like to issue all remaining changes to the database and commit the transaction, whichhas been in progress throughout. We do this via commit():

>>> session.commit()UPDATE users SET password=? WHERE users.id = ?(f8s7ccs, 1)INSERT INTO users (name, fullname, password) VALUES (?, ?, ?)(wendy, Wendy Williams, foobar)INSERT INTO users (name, fullname, password) VALUES (?, ?, ?)(mary, Mary Contrary, xxg527)INSERT INTO users (name, fullname, password) VALUES (?, ?, ?)(fred, Fred Flinstone, blah)COMMIT

commit() flushes whatever remaining changes remain to the database, and commits the transaction. The connectionresources referenced by the session are now returned to the connection pool. Subsequent operations with this sessionwill occur in a new transaction, which will again re-acquire connection resources when first needed.

If we look at Eds id attribute, which earlier was None, it now has a value:

>>> ed_user.idBEGIN (implicit)SELECT users.id AS users_id,


FROM usersWHERE users.id = ?(1,)1

After the Session inserts new rows in the database, all newly generated identifiers and database-generated defaultsbecome available on the instance, either immediately or via load-on-first-access. In this case, the entire row was re-loaded on access because a new transaction was begun after we issued commit(). SQLAlchemy by default refreshesdata from a previous transaction the first time its accessed within a new transaction, so that the most recent state isavailable. The level of reloading is configurable as is described in Using the Session.



Session Object States

As our User object moved from being outside the Session, to inside the Session without a primary key, toactually being inserted, it moved between three out of four available object states - transient, pending, andpersistent. Being aware of these states and what they mean is always a good idea - be sure to read Quickie Introto Object States for a quick overview.

2.1.8 Rolling Back

Since the Session works within a transaction, we can roll back changes made too. Lets make two changes thatwell revert; ed_users user name gets set to Edwardo:

>>> ed_user.name = Edwardo

and well add another erroneous user, fake_user:

>>> fake_user = User(fakeuser, Invalid, 12345)>>> session.add(fake_user)

Querying the session, we can see that theyre flushed into the current transaction:

>>> session.query(User).filter(User.name.in_([Edwardo, fakeuser])).all()UPDATE users SET name=? WHERE users.id = ?(Edwardo, 1)INSERT INTO users (name, fullname, password) VALUES (?, ?, ?)(fakeuser, Invalid, 12345)SELECT users.id AS users_id,


FROM usersWHERE users.name IN (?, ?)(Edwardo, fakeuser)[, ]

Rolling back, we can see that ed_users name is back to ed, and fake_user has been kicked out of the session:

>>> session.rollback()ROLLBACK

>>> ed_user.nameBEGIN (implicit)SELECT users.id AS users_id,


FROM usersWHERE users.id = ?(1,)ued>>> fake_user in sessionFalse

issuing a SELECT illustrates the changes made to the database:



>>> session.query(User).filter(User.name.in_([ed, fakeuser])).all()SELECT users.id AS users_id,


FROM usersWHERE users.name IN (?, ?)(ed, fakeuser)[]

2.1.9 Querying

A Query object is created using the query() method on Session. This function takes a variable number ofarguments, which can be any combination of classes and class-instrumented descriptors. Below, we indicate a Querywhich loads User instances. When evaluated in an iterative context, the list of User objects present is returned:

>>> for instance in session.query(User).order_by(User.id):... print instance.name, instance.fullnameSELECT users.id AS users_id,


FROM users ORDER BY users.id()ed Ed Joneswendy Wendy Williamsmary Mary Contraryfred Fred Flinstone

The Query also accepts ORM-instrumented descriptors as arguments. Any time multiple class entities or column-based entities are expressed as arguments to the query() function, the return result is expressed as tuples:

>>> for name, fullname in session.query(User.name, User.fullname):... print name, fullnameSELECT users.name AS users_name,

users.fullname AS users_fullnameFROM users()ed Ed Joneswendy Wendy Williamsmary Mary Contraryfred Fred Flinstone

The tuples returned by Query are named tuples, and can be treated much like an ordinary Python object. The namesare the same as the attributes name for an attribute, and the class name for a class:

>>> for row in session.query(User, User.name).all():... print row.User, row.nameSELECT users.id AS users_id,


FROM users() ed wendy



mary fred

You can control the names of individual column expressions using the label() construct, which is available fromany ColumnElement-derived object, as well as any class attribute which is mapped to one (such as User.name):

>>> for row in session.query(User.name.label(name_label)).all():... print(row.name_label)SELECT users.name AS name_labelFROM users()edwendymaryfred

The name given to a full entity such as User, assuming that multiple entities are present in the call to query(), canbe controlled using aliased :

>>> from sqlalchemy.orm import aliased>>> user_alias = aliased(User, name=user_alias)

>>> for row in session.query(user_alias, user_alias.name).all():... print row.user_aliasSELECT user_alias.id AS user_alias_id,

user_alias.name AS user_alias_name,user_alias.fullname AS user_alias_fullname,user_alias.password AS user_alias_password

FROM users AS user_alias()

Basic operations with Query include issuing LIMIT and OFFSET, most conveniently using Python array slices andtypically in conjunction with ORDER BY:

>>> for u in session.query(User).order_by(User.id)[1:3]:... print uSELECT users.id AS users_id,


FROM users ORDER BY users.idLIMIT ? OFFSET ?(2, 1)

and filtering results, which is accomplished either with filter_by(), which uses keyword arguments:

>>> for name, in session.query(User.name).\... filter_by(fullname=Ed Jones):... print nameSELECT users.name AS users_name FROM usersWHERE users.fullname = ?



(Ed Jones,)ed

...or filter(), which uses more flexible SQL expression language constructs. These allow you to use regularPython operators with the class-level attributes on your mapped class:

>>> for name, in session.query(User.name).\... filter(User.fullname==Ed Jones):... print nameSELECT users.name AS users_name FROM usersWHERE users.fullname = ?(Ed Jones,)ed

The Query object is fully generative, meaning that most method calls return a new Query object upon whichfurther criteria may be added. For example, to query for users named ed with a full name of Ed Jones, you cancall filter() twice, which joins criteria using AND:

>>> for user in session.query(User).\... filter(User.name==ed).\... filter(User.fullname==Ed Jones):... print userSELECT users.id AS users_id,


FROM usersWHERE users.name = ? AND users.fullname = ?(ed, Ed Jones)

Common Filter Operators

Heres a rundown of some of the most common operators used in filter():

equals:

query.filter(User.name == ed)

not equals:

query.filter(User.name != ed)

LIKE:

query.filter(User.name.like(%ed%))

IN:

query.filter(User.name.in_([ed, wendy, jack]))

# works with query objects too:

query.filter(User.name.in_(session.query(User.name).filter(User.name.like(%ed%))))

NOT IN:

query.filter(~User.name.in_([ed, wendy, jack]))

IS NULL:



filter(User.name == None)

IS NOT NULL:

filter(User.name != None)

AND:

from sqlalchemy import and_filter(and_(User.name == ed, User.fullname == Ed Jones))

# or call filter()/filter_by() multiple timesfilter(User.name == ed).filter(User.fullname == Ed Jones)

OR:

from sqlalchemy import or_filter(or_(User.name == ed, User.name == wendy))

match:

query.filter(User.name.match(wendy))

The contents of the match parameter are database backend specific.

Returning Lists and Scalars

The all(), one(), and first() methods of Query immediately issue SQL and return a non-iterator value.all() returns a list:

>>> query = session.query(User).filter(User.name.like(%ed)).order_by(User.id)>>> query.all()SELECT users.id AS users_id,


FROM usersWHERE users.name LIKE ? ORDER BY users.id(%ed,)[, ]

first() applies a limit of one and returns the first result as a scalar:

>>> query.first()SELECT users.id AS users_id,


FROM usersWHERE users.name LIKE ? ORDER BY users.idLIMIT ? OFFSET ?

(%ed, 1, 0)

one(), fully fetches all rows, and if not exactly one object identity or composite row is present in the result, raises anerror:

>>> from sqlalchemy.orm.exc import MultipleResultsFound>>> try:... user = query.one()



... except MultipleResultsFound, e:

... print eSELECT users.id AS users_id,


FROM usersWHERE users.name LIKE ? ORDER BY users.id(%ed,)Multiple rows were

Documents

SQL Alchemy 0.7.3 manual