Django Testing

Django TestingEric Holscher

http://ericholscher.com

Django Testing

1Tuesday, May 5, 2009



How do you know?

• First 4 months of my job was porting and testing Ellington

• Going from Django r1290 to Django 1.0.

• Suite from 0 to 400 tests.


30,000 Ft View

• State of testing in Django

• Why you should be testing

• How you start testing

• Useful tools

• Eventual Goals


State of Django Testing


assertTrue('Hello World', community.testing.status)


Django 1.1

Making Testing Possible since 2009


manage.py startapp creates a tests.py


from django.test import TestCase

class SimpleTest(TestCase): def test_basic_addition(self): """ Tests that 1 + 1 always equals 2. """ self.failUnlessEqual(1 + 1, 2)

__test__ = {"doctest": """Another way to test that 1 + 1 is equal to 2.

>>> 1 + 1 == 2True"""}


Fast Tests(Transactions)


0

15

30

45

60

Django 1.0 Django 1.1

Ellington Test Speedup

Minutes (Lower is better)


You now have no excuse.


Why to test


Scary13Tuesday, May 5, 2009

Less Scary14Tuesday, May 5, 2009

Not Scary15Tuesday, May 5, 2009

Peace of Mind16Tuesday, May 5, 2009

Code must adapt


“It is not the strongest of the species that survives, nor the most intelligent, but the

one most responsive to change.”

- Charles Darwin


Won’t somebody please think of the users?!


Tests as Documentation


Tests as Documentation

Test driven development +

Document driven development=

Test Driven Documentation


“Code without tests is broken as designed”- Jacob Kaplan-Moss


Dizzying Array of Testing Options


What kind of test?

• doctest

• unittest


Doctests

• Inline documentation

• <Copy from terminal to test file>

• Can’t use PDB

• Hide real failures

• “Easy”


def parse_ttag(token, required_tags): """ A function to parse a template tag.

It sets the name of the tag to 'tag_name' in the hash returned.

>>> from test_utils.templatetags.utils import parse_ttag >>> parse_ttag('super_cool_tag for my_object as obj', ['as']) {'tag_name': u'super_cool_tag', u'as': u'obj'} >>> parse_ttag('super_cool_tag for my_object as obj', ['as', 'for']) {'tag_name': u'super_cool_tag', u'as': u'obj', u'for': u'my_object'}

""" bits = token.split(' ') tags = {'tag_name': bits.pop(0)} for index, bit in enumerate(bits): bit = bit.strip() if bit in required_tags: if len(bits) != index-1: tags[bit] = bits[index+1] return tags


Unit Tests

• Use for everything else

• More rubust

• setUp and tearDown

• Standard (XUnit)


import randomimport unittest

class TestRandom(unittest.TestCase):

def setUp(self): self.seq = range(10)

def testshuffle(self): # make sure the shuffled sequence does not lose any elements random.shuffle(self.seq) self.seq.sort() self.assertEqual(self.seq, range(10))

if __name__ == '__main__': unittest.main()


Django TestCase

• Subclasses unittest

• Fixtures

• Assertions

• Mail

• URLs


Test Client

• Test HTTP Requests without server

• Test Views, Templates, and Context


from django.contrib.auth.models import Userfrom django.test import TestCasefrom django.core import mail

class PasswordResetTest(TestCase): fixtures = ['authtestdata.json'] urls = 'django.contrib.auth.urls'

def test_email_not_found(self): "Error is raised if the provided email address isn't currently registered" response = self.client.get('/password_reset/') self.assertEquals(response.status_code, 200) response = self.client.post('/password_reset/', {'email': '[email protected]'}) self.assertContains(response, "That e-mail address doesn't have an associated user account") self.assertEquals(len(mail.outbox), 0)


mailto:[email protected]

mailto:[email protected]

What flavor of test?

• Unit

• Functional

• Browser


Unit test

• Low level tests

• Small, focused, exercising one bit of functionality


Regression test

• Written when you find a bug

• Proves bug was fixed

• Django Tickets


Functional

• “Black Box Testing”

• Check High Level Functionality


Functional Testing Tools

• Twill

• Django Test Client

• ...


Ellington

• Lots of different clients

• Need to test deployment (Functional)


go {{ site.url }}/marketplace/search/formvalue 1 q pizzasubmitcode 200



Pretty Functional Tests39Tuesday, May 5, 2009

Can test

• All sites on a server

• All sites of a certain type

• A single problemed client

• A test across all sites


Browser tests

• Run tests in a web browser

• Check compatibility of design

• Basically an IE sanity check

• Only real way to test JS, AJAX, CSS


Browser Testing Tools

• Windmill

• Selenium



Other kinds of testing

• Spiders

• Fuzz testing

• Load testing

• Prayer


Where do I start?


Fixed a bug


Poking at code on the command line


Pony turned Horse


Now what?


Start with a regression test or a functional test


Use unittest unless you have a reason not to!


Use the data, Luke

• Use data as a pivot

• Fixtures means you use Unit Tests

• Creation on the command line, Doctests


Creating Fixtures

• ./manage.py dumpdata <app>

• ./manage.py makefixture Model[x:y]

• Follows relations

• By Hand


TestShell

./manage.py testshell <fixture>


Making Functional tests

• Usually a relatively annoying process

• Testmaker makes it easy.

• ./manage.py testmaker <app>

• Simply browse and your session is recorded.


Testing your views generally gets you the

most coverage.


80% Case


When > Where


Tools


Coverage



Mock Objects

• http://www.voidspace.org.uk/python/mock/


http://www.voidspace.org.uk/python/mock/


import unittestfrom mock import Mock from templatetags.cms_tags import if_link_is_active, IsActiveNode class TestIsActiveTag(unittest.TestCase): def test_returns_correct_node_type(self): token = Mock(methods=['split_contents']) token.split_contents.return_value = ('if_link_is_active', 'bar') self.assertEqual(type(if_link_is_active(Mock(), token)), IsActiveNode)


Custom Test Runners


Django Test Extensions

• Gareth Rushgrove

• Extra Assertions

• Coverage and XML Test Runners

• http://github.com/garethr/django-test-extensions


http://github.com/garethr/django-test-extensions

http://github.com/garethr/django-test-extensions

Django Sane Testing

• Ella Folk, Lucas (Hi!)

• Based on nosetests

• Selenium

• Live server

• http://devel.almad.net/trac/django-sane-testing/


http://devel.almad.net/trac/django-sane-testing/

http://devel.almad.net/trac/django-sane-testing/

Django Test Utils

• Mine!

• Testmaker

• Crawler

• Random fanciness

• http://github.com/ericholscher/django-test-utils/tree/master


http://github.com/ericholscher/django-test-utils/tree/master




Recording tests is generally seen as bad.


My philosophy

• Write tests.

• Notice patterns and best practices

• Automate recording of tests with those patterns

• If you can’t automate, use tools to make it easier.


Process for testmaker

• Most view tests check status_code and response context

• Write middleware that catches this info

• Records it to a file

• Err on the side of more data.


Goals(That perfect world)


Some form of TDD

• Write tests as you write code

• Makes your code easy to test


Follow Django’s Model

• Tests with every commit

• Docs with every commit

• Run tests before commiting


Use a DVCS

• At work we have a central SVN repo

• Git feature branches

• Code is staged for documentation and testing

• Committed to SVN once it is “done”


Continuous Integration


NEVER LEAVE THE BUILD BROKEN


Love Green77Tuesday, May 5, 2009

Fast(er) Tests


JSON


Profiling

python -m cProfile manage.py test


Mock Objects





Future and Ponies


Summer of Code

• Test-Only Models

• Coverage

• Windmill tests of the admin


Test Suites in Django


Central Testing Repository


Nose Plugin Integration


Things to remember

• Testing is not hard, you just have to get started.

• If your code doesn’t have tests, it will be hard/impossible to refactor

• Once you have tests, you need to run them!


Credits

• http://www.flickr.com/photos/tym/192416981/

• http://www.flickr.com/photos/seandreilinger/2459266781/

• http://www.homebabysafety.com/images/baby_crawl_suit.jpg

• http://www.flickr.com/photos/pinkypigs/960572985/


http://www.flickr.com/photos/tym/192416981/

http://www.flickr.com/photos/tym/192416981/

http://www.flickr.com/photos/seandreilinger/2459266781/

http://www.flickr.com/photos/seandreilinger/2459266781/

http://www.homebabysafety.com/images/baby_crawl_suit.jpg

http://www.homebabysafety.com/images/baby_crawl_suit.jpg

http://www.flickr.com/photos/pinkypigs/960572985/

http://www.flickr.com/photos/pinkypigs/960572985/

Business

Django Testing