Programming Google App Engine

Dan Sanderson

O'REILLY8

Beijing • Cambridge • Farnham • Köln • Sebastopol • Taipei • Tokyo

Table of Contents

Preface xiii

1. Introducing Google App Engine 1 The Runtime Environment 2 The Static File Servers 4 The Datastore 4

Entities and Properties 5 Queries and Indexes 6 Transactions 6

The Services 8 Google Accounts 9 Task Queues and Cron Jobs 9 Developer Tools 10 The Administration Console 11 Things App Engine Doesn't Do...Yet 12 Getting Started 13

2. Creating an Application 15 Setting Up the SDK 15

Installing the Python SDK 16 Installing the Java SDK 20

Developing the Application 24 The User Preferences Pattern • 24 Developing a Python App 25 Developing a Java App 39 The Development Console 54

Registering the Application 55 The Application ID and Title 57 Setting Up a Domain Name 58 Google Apps and Authentication 59

Uploading the Application 60

VII

Introducing the Administration Console 61

3. Handling Web Requests 63 The App Engine Architecture 64 Configuring the Frontend 66

Configuring a Python App 66 Configuring a Java App 68 Domain Names 69 App IDs and Versions 70 Request Handlers 72 Static Files and Resource Files 75 Secure Connections 81 Authorization with Google Accounts 83

How the App Is Run 85 The Python Runtime Environment 86 The Java Runtime Environment 87 The Sandbox 88 App Caching 89 Logging 93

Quotas and Limits 96 Request Limits 96 CPU Limits 97 Service Limits 98 Deployment Limits 98 Billable Quotas 100 Resource Usage Headers 101

4. Datastore Entities 103 Entities, Keys, and Properties 104 Introducing the Python Datastore API 105 Introducing the Java Datastore API 108 Property Values 110

Strings, Text, and Blobs 112 Unset Versus the Null Value 112 Multivalued Properties 113

Keys and Key Objects 114 Using Entities 116

Getting Entities Using Keys 116 Inspecting Entity Objects 117 Saving Entities 118 Deleting Entities 119

viii | Table of Contents

Datastore Queries 121 Queries and Kinds 122 Query Results and Keys 122 GQL 123 The Python Query API 126

The Query Class 127 GQL in Python 128 Retrieving Results 129 Keys-Only Queries 131

The Java Query API 132 Keys-Only Queries in Java 133

Introducing Indexes 134 Automatic Indexes and Simple Queries 136

All Entities of a Kind 137 One Equality Filter 137 Greater-Than and Less-Than Filters 138 One Sort Order 139 Queries on Keys 141 Kindless Queries 142

Custom Indexes and Complex Queries 143 Multiple Sort Orders 143 Filters on Multiple Properties 144 Multiple Equality Filters 147

Not-Equal and IN Filters 150 Unset and Nonindexed Properties 150 Sort Orders and Value Types 152 Queries and Multivalued Properties 153

A Simple Example 153 MVPs in Python 154 MVPs and Equality Filters 155 MVPs and Inequality Filters 156 MVPs and Sort Orders 157 Exploding Indexes 159

Configuring Indexes 159 Index Configuration for Python * 160 Index Configuration for Java 161

Datastore Transactions 163 Entities and Entity Groups 165

Keys, Paths, and Ancestors 166 Ancestor Queries 167

What Can Happen in a Transaction 168 Transactional Reads 169

Table of Contents | ix

Transactions in Python 169 Transactions in Java 172 How Entities Are Updated 175 How Entities Are Read 178 Batch Updates 179 How Indexes Are Updated 180

7. Data Modeling with Python 183 Models and Properties 184 Property Declarations 185

Property Value Types 186 Property Validation 187 Nonindexed Properties 188 Automatic Values 189 List Properties 190 Models and Schema Migration 191

Modeling Relationships 192 One-to-Many Relationships 195 One-to-One Relationships 195 Many-to-Many Relationships 196

Model Inheritance 198 Queries and PolyModels 199 Creating Your Own Property Classes 200

Validating Property Values 201 Marshaling Value Types 202 Customizing Default Values 204 Accepting Arguments 205

8. The Java Persistence API 207 Setting Up JPA 208 Entities and Keys 209 Entity Properties 212 Embedded Objects 213 Saving, Fetching, and Deleting Objects 214 Transactions in JPA 216 Queries and JPQL 217 Relationships 220 For More Information 225

9. The Memory Cache 227 The Python Memcache API 228

Setting and Getting Values in Python 229 Setting and Getting Multiple Values 230

x | Table of Contents

Memcache Namespaces 231 Cache Expiration 231 Deleting Keys 232 Memcache Counters 233 Cache Statistics 233

The Java Memcache API 234

10. Fetching URLs and Web Resources 239 Fetching URLs in Python 240 Fetching URLs in Java 242 Asynchronous Requests in Python 244

RPC Objects 246 Processing Results with Callbacks 247

11. Sending and Receiving Mail and Instant Messages 251 Enabling Inbound Services 253 Sending Email Messages 254

Sender Addresses 255 Recipients 256 Attachments 257 Sending Email in Python 258 Sending Email in Java 261

Receiving Email Messages 263 Receiving Email in Python 264 Receiving Email in Java 266

Sending XMPP Messages 267 Sending a Chat Invitation 269 Sending a Chat Message 270 Checking a Google Talk User's Status 271

Receiving XMPP Messages 272 Receiving XMPP Messages in Python 273 Receiving XMPP Messages in Java 275

12. Bulk Data Operations and Remote Access 277 Setting Up the Remote API for Python 278 Setting Up the Remote API for Java 279 Using the Bulk Loader Tool 280

Installing SQLite 280 Backup and Restore 281 Uploading Data 282 Downloading Data 286 Controlling the Bulk Loader 289

Using the Remote Shell Tool 290

Table of Contents | xi

Using the Remote API from a Script 291

13. Task Queues and Scheduled Tasks 293 Task Queues 294

Processing Rates and Token Buckets 295 Elements of a Task 296 Task Handlers and Retries 297 Testing and Managing Tasks 299 Using Task Queues in Python 299 Using Task Queues in Java 304 Transactional Task Enqueueing 307

Scheduled Tasks 308

14. The Django Web Application Framework 313 Installing Dj ango 314 Creating a Django Project 315 The Request Handler Script 316 The Django App Engine Helper 317 Creating a Django Application 320 Using App Engine Models With Django 322 Using Django Unit Tests and Fixtures 324 Using Django Forms 327

15. Deploying and Managing Applications 333 Uploading an Application 334 Using Versions 335 Managing Service Configuration 337 Managing Indexes 337 Browsing and Downloading Logs 339 Inspecting the Datastore 342 Application Settings 342 Managing Developers 343 Quotas and Billing 344 Getting Help 345

Index 347

xii | Table of Contents