Testing in Django

testing in django(browser-based testing too)

17 january 2013

Kevin Harvey@kevinharveykevin@storyandstructure.com

Who is this guy?

Application Architect at story+structureDjangonaut since 2007 (version 0.96)

For those of you who didn’t read the bio, I’m Kevin Harvey. I’m the Application Architect at story+structure. We’re based in Brookline and we do software for higher ed. I cut my teeth on Django 0.96 in 2007 with djangobook.com

These are my twin sons Lane & Hank. They are almost 5 months old and awesome.

Who are y’all?

A quick poll...

I’d like to get an idea of who’s in the crowd tonight. If you would please raise your hand if you’ve: - ever started a Python interpreter in Terminal or Command Prompt - ever started a Django project (even just to play around) - gone through the “Polls” app in the Django docs (or feel that you could) - written a single automated test (even just to play around) - released or written a project with a good test suite

What is a test?

An evolutionary perspective

So, what is a test? I’d like to answer that question from an evolutionary perspective.

What is a test?An evolutionary perspective

def multiply_these_two_numbers(a,b):! """! Add 'b' to itself 'a' times! """! i = 0! product = 0! while a > i:! ! product += b! ! i += 1! return product

Let’s say you wrote the function “multiply_these_two_numbers()”, which takes two arguments and adds the second argument to itself by a factor of the first argument. It’s not pretty....

What is a test?An evolutionary perspective

>>> from myapp.my_funcs import multiply_these_two_numbers>>> multiply_these_two_numbers(4,5)20>>> multiply_these_two_numbers(7,8)56>>> multiply_these_two_numbers(720,617)444240

... but it works! You can import it, you can give it two arguments, and it returns what you would expect.

What is a test?An evolutionary perspective

>>> from myapp.my_funcs import multiply_these_two_numbers>>> multiply_these_two_numbers(4,5)20

This is a test. No, it’s not automated, and yes you wrote it by hand in the terminal. But you did confirm that the function: 1) was able to be imported, 2) took two arguments, and 3) returned the expected result.

What is a test?An evolutionary perspective

from myapp.my_funcs import multiply_these_two_numbersimport math

def area_of_rectangle(length, width):! """! Length times width! """! return multiply_these_two_numbers(length, width)!def surface_area_of_sphere(radius):! """! 4 times pi times the radius squared! """! radius_squared = multiply_these_two_numbers(radius, radius)! four_times_pi = multiply_these_two_numbers(4, math.pi)! return multiply_these_two_numbers(radius_squared, four_times_pi)

You tell your colleagues about your new function, and they’re impressed. They start using your function in other parts of the application.

As the code base grows, you’re starting to realize how important your little function is.

What is a test?An evolutionary perspective

from myapp.my_funcs import multiply_these_two_numbers

print multiply_these_two_numbers(4,5)

$ python my_tests.py20$

As you find yourself checking that the function works more and more, you decide to save your test in a file called ‘my_tests.py’ with a print statement to verify the result. Now you can run this saved statement whenever you want and know that your function is working.

What is a test?An evolutionary perspective

>>> import time>>> time.sleep(604800)

Weeks go by. You work on other parts of the project. You work on different projects. You code in different languages.

You have kids. Your mind is erased.

What is a test?An evolutionary perspective

$ python my_tests.py20$

You’ve totally forgotten the context for this test. What does ’20’ mean? You would have to look at the file to see what the inputs were. In short, you don’t know whether this test passed or failed. Can’t we just let Python handle all of this stuff, from setting up the test to remembering what the appropriate output should be?

What is a test?An evolutionary perspective

from django.test import TestCasefrom myapp.my_funcs import multiply_these_two_numbers

class SillyTest(TestCase): def test_multiply_these_two_numbers(self): """ Tests that the function knows how to do math """ self.assertEqual(multiply_these_two_numbers(4,5), 20)

Enter Django’s TestCase with it’s assertEqual() method (based on Python’s own ‘assert’). Here we have a test that knows how to run itself, and knows whether it failed or not.

What is a test?An evolutionary perspective

$ python manage.py test myappCreating test database for alias 'default'....---------------------------------------------------------Ran 1 test in 0.000s

OKDestroying test database for alias 'default'...

Just as a primer, here’s what it looks like when we run that test and it passes. We’ll talk about running tests and the output you get in a moment.

Why write tests?

Why are we doing this?

What’s the point of all these tests? At the outset, it looks like more work: - more code to write - more code to debug... but in fact, we write tests ....

Why write tests?

1. Drink More Beer

To drink more beer, or because we’re lazy. However you want to describe it, we write tests because we want to fix stuff once and only once. Stop worrying about that rickety linchpin of a function holding your entire project together and write a test for it.

Why write tests?

2. Take the fear out of refactoring

Speaking of rickety functions, let’s look back at that awful function we wrote earlier.

Why write tests?UGLY CODE

def multiply_these_two_numbers(a,b):! """! Add 'b' to itself 'a' times! """! i = 0! product = 0! while a > i:! ! product += b! ! i += 1! return product

Bleh. I’d love to fix this, wouldn’t you?

Why write tests?UGLY, VITAL CODE

from myapp.my_funcs import multiply_these_two_numbersimport math

def area_of_rectangle(length, width):! """! Length times width! """! return multiply_these_two_numbers(length, width)!def surface_area_of_sphere(radius):! """! 4 times pi times the radius squared! """! radius_squared = multiply_these_two_numbers(radius, radius)! four_times_pi = multiply_these_two_numbers(4, math.pi)! return multiply_these_two_numbers(radius_squared, four_times_pi)

But it’s used everywhere in our app! What if we screw it up during the refactor?

Why write tests?UGLY, VITAL, TESTED CODE

from django.test import TestCasefrom myapp.my_funcs import multiply_these_two_numbers

class SillyTest(TestCase): def test_multiply_these_two_numbers(self): """ Tests that the function knows how to do math """ self.assertEqual(multiply_these_two_numbers(4,5), 20)

This test (plus a couple more) severely limit the possibility that we’d screw up this function during a refactor. This test is a guarantee that the function, given two integers, will return the product of those two integers.

Why write tests?BETTER CODE

def multiply_these_two_numbers(a,b):! """! Multiply a times b! """#! i = 0#! product = 0#! while a > i:#! ! product += b#! ! i += 1#! return product! return a*b

So refactor...

Why write tests?The test guarantees the refactor

$ python manage.py test myappCreating test database for alias 'default'....---------------------------------------------------------Ran 1 test in 0.000s

OKDestroying test database for alias 'default'...

... and run your tests. If the tests pass, you’re refactor works. Now wouldn’t it be nice if there were tests for all those other functions that your team wrote...

Why write tests?

3. Explain your code

Good tests will serve as technical documentation for your code. You can show your colleagues how your code works by walking them through your tests. And get new developers up to speed quickly by explaining how your tests work.

Why write tests?

4. Clarify your thinking

This is sort of related to the last one, but for me writing tests really help me think through my app. If all my code gets tested, I know I’m only writing the code necessary to get the job done. Testing forces you to write code that’s more modular, which is easier to debug.

Why write tests?

5. Allow more developers to contribute

Think back about the first version of our multiply function: that code was obviously written by a junior developer. Kinda nasty, BUT IT WORKED. It got us from point a to point b. We moved forward because that junior developer contributed a function. We can hack faster because of the test, and we can go back and clean up the mess later.

Why write tests?

6. Be taken seriously

Are you creating something you intend for other developers to use? The first thing I do when evaluating a package from PyPI is check out the tests. It helps me to understand the code, and let’s me know that the package is going to do what it’s supposed to do. It also gives me a glimpse into the way (or whether) the developer thought about the package during development.

Why write tests?

7. Try something out before you screw up your dev environment

This is specific to Django, but I think it’s worth mentioning here.

Why write tests?Keep your dev database clean

$ python manage.py test myappCreating test database for alias 'default'....---------------------------------------------------------Ran 1 test in 0.000s

OKDestroying test database for alias 'default'...

Did you notice the “Creating test database” bit when we ran the test? Django tests create a database from scratch for you every time you run them, and deletes the database when your done. That means you can write tests for new model fields and not have to do the syncdb/migrate/upgradedb dance.

Types of Tests

1. Functional tests2. Unit tests3. Performance tests (which I won’t cover)

The differences between functional tests exist on a series of spectra.

What’s the Difference?

Fewer things(ideally one)

Many things(possibly all)

Unit Functional

Unit tests test a small number of things in your app. Functional tests test a lot.

What’s the Difference?

Small, write many Big, write few

Unit Functional

Unit tests are small and you’ll write a ton. Functional tests are big and you’ll write just a few.

What’s the Difference?

Stuff developerscare about

Stuff userscare about

Unit Functional

Unit tests in general cover things that developers are worried about. Functional tests test things users care about.

What’s the Difference?

Fast Slow

Unit Functional

Unit tests run fast (thousandths of a second). Functional tests run slow (seconds or more).

A few examples

Django tests in action!

Unit test for Model

def test_questions_increment_votes_up(self):! """! Test voting up a question! """! question_1 = Question(text="How can my team get started?",! ! ! ! ! ! votes=7)! ! ! ! ! !! question_1.increment_votes(4)!! self.assertEquals(question_1.votes, 11)

A few examples

Unit testing a model.

Unit test for Model

class Question(models.Model):! text = models.CharField("Question", max_length=500)! votes = models.IntegerField()!! def increment_votes(self, num):! ! self.votes += num! ! self.save()

A few examples

This code makes the previous test pass.

Unit test for a Form

def test_question_form_excludes_all_but_text(self):! """! The user can only supply the text of a question! """! form = QuestionForm()! self.assertEquals(form.fields.keys(), ['text'])! self.assertNotEquals(form.fields.keys(),

['text', 'votes'])

A few examples

Unit test for a Form

class QuestionForm(forms.ModelForm):! class Meta:! ! ! model = Question! ! ! fields = ('text',)

A few examples

Make the previous form unit test pass.

Unit test for a POST action

def test_ask_question(self):! """! Test that POSTing the right data will result in a new question

! """! response = self.client.post('/ask/',

{'text': 'Is there any more pizza?'})! !! self.assertRedirects(response, '/')! !! self.assertEqual(Question.objects.filter(text='Is there any more pizza?').count(), 1)

A few examples

Unit test for a POST action

def ask(request):if request.method == "POST":

! question_form = QuestionForm(request.POST)! question_form.save()return HttpResponseRedirect('/')

A few examples

urlpatterns = patterns('',...

url(r'^ask/$', 'questions.views.ask', name='ask'),...

You have to edit two files to get the previous test to pass.

Functional test: Logging in

def test_admin_can_manage_questions(self):

self.browser.get(self.live_server_url + '/admin/')! ! username = self.browser.find_element_by_css_selector("input#id_username") username.clear() username.send_keys("peter") password = self.browser.find_element_by_css_selector("input#id_password") password.clear() password.send_keys("password") self.browser.find_element_by_css_selector("input[type='submit']").click()

body = self.browser.find_element_by_tag_name('body') self.assertIn('Site administration', body.text)

A few examples

This test uses Selenium to test that an admin user is able to login in to the admin site.

Functional test: Logging in

# set up the admin site

A few examples

Just set up the admin site to get it to pass.

Test Driven Development

What is Test Driven Development?

Simply put, TDD means writing tests that fail and THEN writing the code to make them pass. Guards against code explosion because you only write enough code to make the test pass.

TDD by Example

Our example project: Torquemada

TDD by Example

http://bit.ly/XHjcAior

http://infinite-meadow-8366.herokuapp.com/

Codehttps://github.com/kcharvey/testing-in-django

or use the link at the demo

See the demo at either of the first URLs. The code is available on GitHub.

TDD by Example

Torquemada allows attendees at “Testing in Django (browser based testing too)” to inquire of the presenter asynchronously.

Our example project: Torquemada

http://bit.ly/XHjcAi

A description of the app we’ll be building.

TDD by Example

Use Case: Isabella the Inquisitive

Isabel has learned a lot by attending the “Testing in Django” MeetUp but still has a few questions for the presenter. She visits Torquemada in her web browser, where she can see if any of her inquiries have been addressed and see the question that is being currently discussed. She is able to ‘vote up’ questions she would like the presenter to answer, and ‘vote down’ questions she thinks are unimportant. She is also able to ask her own question.

Our example project: Torquemada

http://bit.ly/XHjcAi

A use case for a user.

TDD by Example

Use Case: Peter the Presenter

Peter is presenting at the “Testing in Django” MeetUp and would like to answer any questions the attendees may have during set periods in the talk. He views Torquemada in his web browser to see what questions attendees have asked, including relative importance based on the number of votes they’ve received. When the group is discussing a question, he uses the Django admin site to set the question’s status to “Current”. After the question has been discussed, he sets the status to “Archived”

Our example project: Torquemada

http://bit.ly/XHjcAi

A use case for an admin.

Let’s do this.

TDD by example

http://vimeo.com/57692050

Screencast 1

(watch the video, then come back for the rest of the slides)

1) Set up a workspace 2) create a virtualenv 3) install django 4) startproject 5) startapp 6) test the app. You can start TDD on a Django project without even touching settings.py!

TDD by example

Text

from django.test import TestCase

class SimpleTest(TestCase): def test_basic_addition(self): """ Tests that 1 + 1 always equals 2. """ self.assertEqual(1 + 1, 2)

The test that ‘manage.py startapp’ generated.

Texthttp://vimeo.com/57693303

Screencast 2

Replace the auto generated test with a meaningful one.

TDD by example

Text

from django.test import LiveServerTestCasefrom selenium import webdriver

class QuestionsTest(LiveServerTestCase):

def setUp(self): self.browser = webdriver.Firefox() self.browser.implicitly_wait(3)

def tearDown(self): self.browser.quit()...

Start with a functional test that tells (at least part of) a use case.

TDD by example

Text

...class QuestionsTest(LiveServerTestCase):...def test_can_read_v..._a_question(self):# Isabel opens her web browser and# visits Torquemadaself.browser.get(self.live_server_url + '/')

# TODO self.fail('finish this test')

Texthttp://vimeo.com/57692852

Screencast 3

Extend the functional test with some use case as comments, and add a test for an h1 element.

TDD by example

Text# She knows it's Torquemada because she sees the name in the headingheading = self.browser.find_element_by_css_selector("h1#trq-heading")self.assertEqual(heading.text, "Torquemada")

Here’s the assertion, using Selenium.

Texthttps://vimeo.com/57693096

Screencast 4

Watch the new version of the functional test fail.

We have a working (but failing) test.Commit it.

TDD by example

Meaningful failure = important unit of work we’ve done. Let’s commit it.

Let’s refactor a bit...

TDD by example

Texthttp://vimeo.com/57693617

Screencast 5

Refactor the test.py file into a tests package.

... and commit that.

TDD by example

So, where are we exactly?

TDD by example

Texthttp://vimeo.com/57694735

Screencast 6

In the final screencast, we finish up this round of TDD: dive into a unit test for a view, write the code to get the test to pass, extend the unit test to cover more functionality, get it to pass, then confirm that we’ve satisfied our functional test (as it is).

Rinse and repeat.

TDD by example

Repeat the cycle of writing tests that fail (or extending tests to make them fail) and writing code to make the tests pass.

How about a demo?

Getting Around with Selenium

self.browser.get()

This command (if you’ve set self.browser = webdriver.FireFox(), or something like it), does a GET to open a web page with Selenium. It can open any URL.

Getting Around with Selenium

self.browser.find_element_by_<METHOD>()

Find elements on the page using one of Seleniums ‘find_element’ methods.

Getting Around with Selenium

find_element_by_idfind_element_by_namefind_element_by_xpathfind_element_by_link_textfind_element_by_partial_link_textfind_element_by_tag_namefind_element_by_class_namefind_element_by_css_selector # use this one

find_element_by_css_selector is my favorite. It feels like jQuery.

Getting Around with Selenium

find_elements_by_idfind_elements_by_namefind_elements_by_xpathfind_elements_by_link_textfind_elements_by_partial_link_textfind_elements_by_tag_namefind_elements_by_class_namefind_elements_by_css_selector

# return Python lists

You can get lists of elements by pluralizing ‘elements’ in the method name.

Getting Around with Selenium

element.click()

Once you find an element, you can .click() it. This will behave just like a real user click, and respects an JavaScript event.preventDefault() handlers.

Getting Around with Selenium

text_field = self.browser.find_element_by_css_selector("#id_text")text_field.clear()text_field.send_keys("Why aren't you using reverse()?")

self.browser.find_element_by_css_selector("input#trq-submit-question").click()

Working with form elements

Use .clear() and .send_keys() to type test into a form element.

How do I get started?

I want to start testing on my team. What’s the best way?

How do I get started?

1. Write tests for bug reports

An excellent way to start is to write tests for your bug reports. Find out the exact steps it takes to reproduce a problem and write either a functional or unit test THAT FAILS as an illustration of the bug. Then, fix the bug. Check in your fix and your test and rest easy knowing you’ve GUARANTEED that the bug is fixed.

How do I get started?

2. Use TDD for new features

There’s no time like the present to start writing tests. When your team decides on a new feature to implement, start by writing some use cases and develop from a functional test.

How do I get started?

3. Write unit tests for existing code you use while doing 1. and 2.

If you’re using a function from somewhere else in the system when you write code, you need to be able to guarantee that function does what you expect it to. Get in the habit of writing tests for parts of your project as you come in to contact with them, PARTICULARLY if they are involved in a bug fix.

What about continuous integration?

Continuous integration tools automatically checkout our code, build it, and run the tests. It’s to protect ourselves from developers on our team that forget to run the tests before they commit.

What about continuous integration?

We’re trying out Jenkins.

s+s is just starting to play with Jenkins, a CI platform in written in Java. NORMALLY, I’M LIKE...

.WAR

WHAT

IS

IT

GOOD

FOR?

What about continuous integration?

But in this case I’ll make an exception.

Jenkins is super easy to get up and running on your local machine, and there are plugins that play nice with tools we’re all using.

Configuring Jenkins$ pip install django-jenkins

INSTALLED_APPS = ( ... 'django_jenkins',)...

JENKINS_TASKS = ( 'django_jenkins.tasks.run_pylint', 'django_jenkins.tasks.with_coverage', 'django_jenkins.tasks.django_tests', # there are more of these)

$ python manage.py jenkins # Jenkins will run this command

django-jenkins is a plugin that runs out tests and outputs the files Jenkins needs to show our build stats. pip install and add just a few lines to your settings.py

Configuring Jenkins

$ wget http://mirrors.jenkins-ci.org/war/latest/jenkins.war$ java -jar jenkins.war

http://localhost:8080

Just get the .war file, run it, and hit port 8080 on your machine.

Configuring JenkinsYou’ll need some plugins:

- Jenkins Violations Plugin- Jenkins Git Plugin- Jenkins Cobertura Plugin

Install a few plugins.

Configuring Jenkins

1. Configure a new test (name, description)2. Give it your repo URL3. Tell it how often to build4. Tell it the commands to run5. Configure where to save the reports6. Click “Build Now”

Check out the tutorials in the “Resources” section of this slide deck for more on configuring your repo. It’ll take about 15 minutes the first time.

Configuring Jenkins

DONE

That’s it.

Next Steps508 compliance

http://wave.webaim.org/toolbar

JavaScript Testing

http://doctestjs.org/

Doctest.js

There are things we definitely didn’t test. We’re looking into automated 508 compliance testing. If you’re working in higher ed or the non-profit world, a non-compliant template should definitley “break the build”.

Resources

• http://www.tdd-django-tutorial.com/

• http://toastdriven.com/blog/2011/apr/10/guide-to-testing-in-django/ (and part 2)

•http://code.google.com/p/selenium/wiki/IPhoneDriver

Tutorials

Resources

• https://github.com/kmmbvnr/django-jenkins

• https://sites.google.com/site/kmmbvnr/home/django-jenkins-tutorial

• http://jenkins-ci.org/

Jenkins

Resources

• http://funkload.nuxeo.org/

• http://jmeter.apache.org/

• http://httpd.apache.org/docs/2.2/programs/ab.html

Performance Testing

Using fixtures

Manage data for tests runs

Sometimes you want to create models explicitly. Other times you want to create a lot of reusable data. Enter fixtures.

Using fixtures[ { "pk": 1, "model": "questions.question", "fields": { "status": "new", "text": "How can my team get started with testing?", "votes": 0, "created": "2013-01-17T16:15:37.786Z" } }, { "pk": 2, "model": "questions.question", "fields": { "status": "new", "text": "Does Selenium only work in Firefox?", "votes": 0, "created": "2013-01-17T16:17:48.381Z" } }]

A fixture is just structured data (defaulting to JSON) that Django knows how to import and export: - loaddata - dumpdata

Using fixtures

$ python manage.py runserver...

# Use the Django /admin site to make some test data

...

$ mkdir questions/fixtures/$ python manage.py dumpdata questions --indent=4 > questions/fixtures/questions.json

Do NOT try to write fixtures by hand

Use the admin site and ‘manage.py dumpdata’ to make fixtures easily.