Testing with Python

The REPL(aka the Python Interpreter)

Impress your friends:

Read

Eval

Print

Loop

Pros

Familiar to Pythonistas

Great for exploration

Simple to use

Cons

Manual

Error-prone

Doesn't scale

We need automation!

And lo, there were Doctests!

def double(value): """ >>> double(1) 2 >>> double(2) 4 >>> double("foo") 'foofoo' >>> double(None) Traceback (most recent call last): File "", line 1, in File "", line 2, in double TypeError: unsupported operand type(s) for *: 'NoneType' and 'int' """

return value * 2

def double(value): """ >>> double(1) 2 >>> double(2) 4 >>> double("foo") 'foofoo' >>> double(None) Traceback (most recent call last): File "", line 1, in File "", line 2, in double TypeError: unsupported operand type(s) for *: 'NoneType' and 'int' """

return value * 2

A REPL session "embedded" in a function's doc string

Pros

Automated

All the benefits of the REPL

Code and tests together

Cons

Code and tests together

Can get big and clunky

Not very flexible

We need something better!

Behold: Unittest!

Unittest is part of the xUnit family

xUnit started with SUnit for Smalltalkand has become a de facto standard

The API is broadly similar acrossmany different languages.

Pros

Automated

Flexible

Scalable

Part of the standard library

Part of the xUnit family

Cons

"Unpythonic" Java-esque API

Neglected until recently

Code and tests separate (pro?)

Potential for unwieldy test suites

The Building Blocks

Assertions

Test Cases

Test Fixtures

Test Suites

Test Runners

Test Runners

A test runner is a component which orchestrates theexecution of tests and provides the outcome to theuser. The runner may use a graphical interface, atextual interface, or return a special value to indicatethe results of executing the tests.

Source: Python documentation

Test Suites

A test suite is a collection of test cases, test suites,or both. It is used to aggregate tests that should beexecuted together.

Source: Python documentation

Test Fixtures

A test fixture represents the preparation needed toperform one or more tests, and any associate cleanupactions. This may involve, for example, creatingtemporary or proxy databases, directories, or startinga server process.

Source: Python documentation

Test Cases

A test case is the smallest unit of testing. It checks fora specific response to a particular set of inputs. unittestprovides a base class, TestCase, which may be usedto create new test cases.

Source: Python documentation

Assertions

In computer programming, an assertion is a predicate(for example a truefalse statement) placed in aprogram to indicate that the developer thinks that thepredicate is always true at that place.

Source: Wikipedia

import unittest

class TestFoo(unittest.TestCase): def test_something(self): self.assertTrue('x' in 'xyz')

if __name__ == '__main__': unittest.main()

import unittest

class TestFoo(unittest.TestCase): def test_something(self): self.assertTrue('x' in 'xyz')

if __name__ == '__main__': unittest.main()

This calls the standard test runner, and gathersup all TestCase classes in the file.

import unittest

class TestFoo(unittest.TestCase): def test_something(self): self.assertTrue('x' in 'xyz')

if __name__ == '__main__': unittest.main()

Test cases are subclasses of TestCase.

import unittest

class TestFoo(unittest.TestCase): def test_something(self): self.assertTrue('x' in 'xyz')

if __name__ == '__main__': unittest.main()

And there's an assertion.Simples.

There are more types of assertionthan assertTrue, otherwise thingscould get awkward.

Note: I'm still stuck with Python 2.4,2.5 and 2.6!

Python 2.7 and 3.x have lots of newassertions available.

assertTrue has a negative counterpart:

assertTrue(x)assertFalse(x)

...which can aid readability, but isn'tparticularly exciting.

We can test for equality:

assertEqual(x, y)

Which is asserting that x == y

Like assertTrue, it has a counterpart:

assertNotEqual(x, y)

A personal favourite of mine is:

fail(message)

Which I use as a placeholder:

fail("Test not implemented")

fail("Good. You do run the tests")

Ah, yes... what happens if a test fails?

Speaking of messages, most assertions havean optional argument for a message.

assertTrue(x, "It lied")assertEqual(x, y, "Blah blah")

The message shows if the assertion fails.

Another important assertion is:

assertRaises(Exception, foo.x, y)

In newer versions of Python, this has a muchcleaner syntax...

with self.assertRaises(Exception):foo.x(y)

Keep tests simple

Test one thing at a time

class TestFoo(unittest.TestCase): def test_double_one(self):foo = MyFoo()

self.assertEqual(2, foo.double(1))

def test_double_two(self):foo = MyFoo()

self.assertEqual(4, foo.double(2))

Did you spot the duplication?

Time to introduce Test Fixtures...

Fixtures allow us to set up and teardown the environment of the tests.

It also allows us to factor out anyboilerplate and duplication.

class TestFoo(unittest.TestCase): def test_double_one(self):foo = MyFoo()

self.assertEqual(2, foo.double(1))

def test_double_two(self):foo = MyFoo()

self.assertEqual(4, foo.double(2))

Let's factor out the duplication...

class TestFoo(unittest.TestCase): def setUp(self): self.foo = MyFoo()

def test_double_one(self): self.assertEqual(2, self.foo.double(1))

def test_double_two(self): self.assertEqual(4, self.foo.double(2))

class TestFoo(unittest.TestCase): def setUp(self): self.foo = MyFoo()

def test_double_one(self): self.assertEqual(2, self.foo.double(1))

def test_double_two(self): self.assertEqual(4, self.foo.double(2))

setUp is called before every test in the test case.

setUp has a partner called tearDownwhich, if present, is called after everytest.

Ideal for cleaning up afterwards.

def tearDown(self): self.foo.close()

Things to think about when testing...

Keep each test case focused on aparticular unit of code

Organise tests within the test caseto check individual aspects of the code

Helps keep the tests simple andmaintainable

Multiple test cases can exist in a file.

Ideally, break out the test cases intomultiple files just as you would withyour code.

If code is difficult to test: refactor orrewrite to make it easier.

This also leads to simpler, moremaintainable code.

When you write tests, you're addingconfidence in the code

There's always one more bug to findThere's always one more test to write

If you find a bug in your code, writea test to replicate it before youimplement a fix

Even a "bad" test gives value overno test.

Bad tests can always be improved aswe understand the problem / solution.

Code under test should be isolatedwhenever possible, to minimiseexternal influence

We can isolate code from things likedatabases, network connections, etc.through Fakes and Mocks

Fakes are dummy objects that donothing except stand-in for something

Mocks are more interesting. They cancapture and respond to messages

And now over to Michael...