Large problems, Mostly Solved

Large ProblemsMostly Solved

Eric Holscherhttp://ericholscher.com

Djangocon 2010

Wednesday, September 8, 2010

What this talk isn’t

» In depth

» Overly technical

» Conclusive

What this talk is

» Providing current best-of-class solutions to common problems

» Identifying problems that aren’t currently solved well

Overview

» Patterns among good apps

» The problems that are mostly solved

» Unsolved Problems

A good application

Leverages previous knowledge/APIs

Easy Setup

Good upgrade path

Good documentation

Well Tested

Solves a real problem

Mostly Solved Problems

Search

Haystack

» 9782 lines of docs

» 8882 lines of tests

» http://haystacksearch.org/

» Written by Daniel Lindsley

Line counts

find . |egrep "rst|txt" |xargs wc -l |cut -c1-8 | awk '{s+=$1} END {print s}'

find . |egrep "py" |xargs wc -l |cut -c1-8 | awk '{s+=$1} END {print s}'

#FORE!

Setup» pip install django-haystack

» Add ‘haystack’ to Installed Apps

» Settings for siteconf and backend

» Create Search Site

» Create Search Indexes

» Add views

» Reindex

Upgrade Path

» Simple backend

» Whoosh

» Solr

Beautiful APIs

unfriendly_results = SearchQuerySet().exclude(content='hello').filter(content='world')

unfriendly_results.order_by('-pub_date')[:5]

Beautiful APIs

class NoteIndex(SearchIndex): text = CharField(document=True, use_template=True) author = CharField(model_attr='user') pub_date = DateTimeField(model_attr='pub_date')

def get_queryset(self): return Note.objects.filter(pub_date__lte=datetime.datetime.now())

Awesome Features

» Faceting

» Highlighting

» More Like This

» Easy Customization

Amazing full-text search

In an hour.

Documentation

Sphinx

» http://sphinx.pocoo.org/

» Written by Georg Brandl

» pip install Sphinx

» sphinx-quickstart

» Edit/Create Restructured Text files

» make html

Awesome features

» Uses ReStructuredText

» Easily generate PDFs

» Link between your own and other docs

» Themes

Shameless Self Promotion

» Use Read The Docs

» Host Sphinx docs, or create them there.

» http://rtfd.org

» http://media.readthedocs.org/django.pdf

Make Beautiful Documentation With Tools

You Know

Database Migrations

» http://south.aeracode.org/

» Written by Andrew Godwin

» pip install south

» south in Installed Apps

» manage.py schemamigration app --initial

» manage.py schemamigratoin app name --auto

» manage.py migrate

Beautiful API

def forwards(self, orm): for adopter in orm.Adopter.objects.all(): try: adopter.first_name, adopter.last_name = adopter.name.split() except ValueError: adopter.first_name, adopter.last_name = adopter.name, "" adopter.save()

Boring

Safe, Painless Data Migration

Delayed Execution

Run commands out of process

Celery

» http://celeryproject.org/

» Written by Ask Solem

» pip install django-celery

» Add ‘djcelery’ to Installed Apps

» Syncdb

» Configure your settings backend

» Add @task decorator to function

When and Why

» Makes your site fast

» Use for tasks that users don’t immediately care about

» Even if they do, return fast to a waiting page

Examples

» Activity

» Sending Email

» Complex DB queries

» Start with ALWAYS_EAGER

» Use a ghetto queue

» Rabbit MQ

Awesome features

» Run on multiple machines

» Cron replacement

» Good error reporting

Magically make your code async

Remote Execution

Run commands on remote servers (for deployment)

Fabric

» http://fabfile.org

» Written by Christian Vest Hansen & Jeff Forcier

» pip install fabric

» Add commands to a fabfile.py

» fab <command>

Simple example

#Fabfilefrom fabric.api import run

def host_type(): run('uname -s')

# Command$ fab -H example.com host_type

Write python to deploy your code

Deployment

Gunicorn

» http://gunicorn.org/

» Created by Benoit Chesneau

» pip install gunicorn

» add gunicorn to your Installed Apps

» manage.py run_gunicorn

Need to install Nginx in front

Nginx setup

server { listen 80; server_name example.com; access_log /var/log/nginx/example.log;

location / { proxy_pass http://127.0.0.1:8080; }}

Awesome Features

» Async backend workers

» Simple configuration

» Python

Stupidly Simple, Production Ready

Deployment

Packaging

Pip & Virtualenv

» Written by Ian Bicking

» http://pip-installer.org

» http://virtualenv.openplans.org

Set up

» sudo easy_install pip # :D

» sudo pip install virtualenv

» virtualenv awesome_env

» source awesome_env/bin/activate

» pip install Django

Awesome Features

» Separate environments for each application or deployment

» Install code from repos

» Doesn’t use eggs

» User-owned environment

Adds community standards to something everyone was

doing their own way

No more PYTHONPATH hacking

Competing Applications

» TastyPie

» Piston

Tagging

» Django Taggit *

» Django Tagging

Continuous Integration

» Hudson *

» Buildbot

Smaller Problems

Debugging in Development

django-debug-toolbar

» http://github.com/robhudson/django-debug-toolbar/

» Written by Rob Hudson

Profiles & Registration

django-[profiles|registration]

» http://bitbucket.org/ubernostrum/django-profiles/

» http://bitbucket.org/ubernostrum/django-registration/

» Written by James Bennett

Filtering

django-filter

» http://github.com/alex/django-filter

» Written by Alex Gaynor

Finding packages

Djangopackages.com

Unsolved Problems

Template Tags

Logging

Model Introspection

Class Based Views

Source code arrangement

Notifications/Email

Debugging in Production

OpenID/OAuth

VCS Abstraction

Positive Note

Cal’s list of problems ’08» Multiple Databases

» Multiple multi-keyed cache requests

» Slow HTTP requests while we're doing other stuff

» ORM creates dumb SQL

» Verbose template syntax

» No query debugger

» Ugly SQL syntax

» Commit all fields at once

» No mascot

» No deployment system

» Migrating ModelsWednesday, September 8, 2010

Cal’s list of problems ’08» Multiple Databases

» Multiple multi-keyed cache requests

» Slow HTTP requests while we're doing other stuff

» ORM creates dumb SQL

» Verbose template syntax

» No query debugger

» Ugly SQL syntax (Table aliases)

» Commit all fields at once

» No mascot

» No deployment system

» Extending modelsWednesday, September 8, 2010

Remember

» Leverage good applications in the community

» Take previous knowledge and apply it to new situations

» You can create an awesome application used by other people

Questions?

» eric@ericholscher.com

» twitter.com/ericholscher

Large problems, Mostly Solved

Documents

G3- Solved Problems

Adc Solved Problems

Ch4 Solved Problems

AI FOR MATTERS - GitHub Pages · Machine learning settings. Supervised learning (mostly machine) A B. Unsupervised learning (mostly human) Will be quickly solved for “easy” problems

Solved Problems in Soil Mechanics - site.iugaza.edu.pssite.iugaza.edu.ps/.../Solved-Problems-in-Soil-Mechanics...Update.pdf · Soil Properties & Soil Compaction Page (1) Solved Problems

2.9 Problems Solved

Problems Solved and Not Solved in UCG

Calculus Solved Problems

Additional Solved Problems

(MOSTLY) SIMPLE (MOSTLY AREA) PROBLEMS · (MOSTLY) SIMPLE (MOSTLY AREA) PROBLEMS Tatiana Shubin shubin@math.sjsu.edu I. EASY PROBLEMS 1. (AJHSME 1995) Figures I, II and III are squares

Financial Management Solved Problems

3,000 Solved Problems in Physics (Schaum's Solved Problems) (Schaum's Solved Problems Series)

Fsa Solved Problems

Dog Problems Solved - demonx.orgdemonx.org/mom/Dog-Problems-Solved.pdf · Dog Problems Solved! - :: -

Communication Solved Problems

Solved Problems

Matlab solved problems

corrosion solved problems

Solved Problems- Chap29

3000 Solved Problems