Emily Bache - Unit Testing Python

Unit Testing with Python

Emily Bachehttp://coding-is-like-cooking.info

[email protected]

http://coding-is-like-cooking.info


mailto:[email protected]


Course Summary

Unit testing vocabulary & basic example using unittest Unit testing - Why and When An alternative to unittest: pytest

Testable documentation using doctest Test Doubles Assessing Test Coverage Maintainable Unit Tests

“Goat in Tree” - Cuno de Boer (!ickr)

Module Summary

Unit Testing vocabulary & “Phonebook” example Test Case design


Review of Fundamentals

System Under Test

a Unit Test checks the behaviour of an element of code a method or function a module or class

an automated test is designed by a human runs without intervention reports results unambiguously as “pass” or “fail”

Strictly speaking

It’s not a unit test if it uses... the !le system a database the network

(but it might still be a useful test)

System Under Test

Exercise - Phone Numbers

Image aribution: icons8.com, Aha-Soft

Given a list of names and phone numbers, make a Phonebook that allows you to look up numbers by name.

Determine if a given Phonebook is consistent. In a consistent phone list no number is

a pre!x of another. For Example: Bob 91125426 Alice 97 625 992 Emergency 911 Bob and Emergency are inconsistent

Unit test vocabulary: TestCase

Test Case System Under Test

Unit Test Vocabulary


Test Runner



Test Runner

or



Test Runner

Test Suite



Test Runner

Test Suite

Test Fixture

Test Fixture

Test Fixture - Test Fails

Test Fixture - Setup Fails



Test Runner

Test Suite

Test Fixture

Test Case Design

• Test Case Name• Arrange• Act• AssertTest Case

The Three Parts of a Test

Arrange: Set up the object to be tested & collaborators.

Act: Exercise functionality on the object.

Assert: Make claims about the object & its collaborators

Test Case

The Three Parts of a Test

Arrange: Set up the object to be tested & collaborators.

Act: Exercise functionality on the object.

Assert: Make claims about the object & its collaborators

Test Case

Cleanup: Release resources, restore to original state

Four

Module Review

Unit Testing vocabulary Test Case Test Runner Test Suite Test Fixture

Test Case design Test name Arrange - Act - Assert


Unit Testing with Python


[email protected]





Module Overview

What is unit testing for? Unit testing in your personal

development process. Test First Test Last Test Driven

Unit testing in the wider Development Process

What is Unit Testing For?

Test Suite

Image aribution: semlabs.co.uk, FatCow

Understand what to build

Design the Units

Document the Units

Regression Protection

Understanding

Collaborate with people in other roles to understand what’s needed Business Analyst, Product Owner Tester Interaction Designer Lead Developer


Documenting

“Executable Speci!cation” Tests document the behaviour of the code How the unit is intended to be used

Document the Units

Design

Decompose the problem into units that are independently testable loose coupling

Design the Interface separately from doing the Implementation

Design the Units


Regression - something worked before and doesn’t any more A Unit test should fail & point out which unit failed and why



Test Suite



Design the Units

Document the Units


Limitations of Unit Testing

Testing can’t !nd all the errors Unit testing won’t !nd integration errors

Test First, Test Last, Test Driven

Test Suite



Design the Units Regression Protection

Document the Units

Your personal Development Process

Test SuiteProduction Code

Test First, Test Last, Test Driven

Test Last

Test Last

Design code

Design testsDebug & Rework

Risk: Discover testability problems and bugs late in the process

Risk: You’ll rush or skip designing the tests

Test FirstTest First

Design tests

Write code

Design code

Refactor code

Refactor tests

Risk: Rework

Test Driven

Write a test

Test Driven

Write a lile codeRefactor

Unit testing in the wider Development Process

Version Control

Version Control

Production Code Test Suite

Work from Known Good

Version Control

Unexpected Failures

Version Control

Continuous Integration

Icons by FatCow, Double-J Design

Version Control

Continuous Integration

Server

See also Martin Fowler’s article: http://www.martinfowler.com/articles/continuousIntegration.html

http://www.martinfowler.com/articles/continuousIntegration.html

http://www.martinfowler.com/articles/continuousIntegration.html

Coverage metrics

Module Review

What is unit testing for? Test First, Test Last, Test Driven Unit testing in the wider Development

Process

Unit Testing with PythonModule 3: pytest


[email protected]





Module Overview

De!ning test cases with py.test Interpreting Test Failures Test Fixtures

xUnit

Kent BeckErich Gamma

JUnit was originally created by Kent Beck and Erich Gamma

Nose and Pytest

hps://code.google.com/p/python-nose/

hp://pytest.org/latest/

https://code.google.com/p/python-nose/

https://code.google.com/p/python-nose/

http://pytest.org/latest/

http://pytest.org/latest/

http://pytest.org/latest/getting-started.html#installation



Test Runner

Test Suite

Test Fixture

Test Fixture Functions

Test Case Test FixtureRequest a resource

?

Test Runner

Test Fixture - cleaning up


Test Runner

Test Suite

Test Fixture

Test Fixture Functions

Test Case Test FixtureRequest a resource

?

Test Runner

Request a resource

?

Unit Tests should be In Memory

Test Case File System

icons: Double-J Design

Integration Tests may need test "xtures that provide slow and expensive subsystems

Pytest + unittest


Test Runner

Test Suite

Test Fixture

Pytest + unittest


Test Runner

Test Suite

Test Fixture

Module Review

De!ning test cases with py.test Interpreting Test Failures Test Fixtures

Unit Testing with PythonModule 4: doctest


[email protected]





Module Outline

Situations when you’d use doctest Making documentation comments

more truthful Handling output that changes using doctest for regression testing

doctest

Checking examples in docstrings. Regression testing. Tutorial documentation.

Making documentation comments more truthful

Docstrings can get out of date Doctest helps you keep them updated


Test Suite



Design the Units

Document the Units


Yatzy

Roll 5 dice (and re-roll some) choose a category to score the roll in Each category is only used once The !nal score is the sum of the score in each category

image by !"වාIcons: FatCow, Aha-Soft

http://commons.wikimedia.org/wiki/User:%E0%B6%B6%E0%B7%92%E0%B6%9F%E0%B7%94%E0%B7%80%E0%B7%8F

http://commons.wikimedia.org/wiki/User:%E0%B6%B6%E0%B7%92%E0%B6%9F%E0%B7%94%E0%B7%80%E0%B7%8F

Yatzy example

“Two Pairs” Score: 2 + 2 + 3 + 3 = 10

“Threes” Score: 3 + 3 = 6

“Small Straight” Score: 0

Handling output that changes

Handling output that changes

Dictionaries Floating point numbers Object ids Tracebacks?

Doctest Directives

Directives control how doctest matches output Use wildcard matching with care

hp://docs.python.org/3.3/library/doctest.html#doctest-directives

http://docs.python.org/3.3/library/doctest.html#doctest-directives

http://docs.python.org/3.3/library/doctest.html#doctest-directives

doctest


Approval Testing

“I’ll know it when I see it”

Write a test

Write a lile codeRefactor

Approval Testing

“I’ll know it when I see it”

Arrange a test

Write a lile code

Approve a result

Refactor


Test Suite



Design the Units

Document the Units


What role is your doctest playing?


Document the Units


Write it in a docstring

Put it in a separate "le& consider moving to uniest or pytest

doctest


Module Review

Situations when you’d use doctest Checking examples in docstrings Regression Testing Tutorial documentation

Handling output that changes using pytest as a test runner for

doctests

Unit Testing with PythonModule 5: Test Doubles


[email protected]





Module Overview

What is a Test Double? Different kinds of Test Double Why use Test Doubles? using Monkeypatching to insert

Test Doubles

Test Double

like “Stunt Doubles” who stand in for actors in "lms

class under test doesn’t know it isn’t talking to the real object

Allow you to control what happens to your class under test

Different kinds of test double

from http://xunitpatterns.com

see also: hps://en.wikipedia.org/wiki/Test_Double

http://xunitpatterns.com

http://xunitpatterns.com

https://en.wikipedia.org/wiki/Test_Double

https://en.wikipedia.org/wiki/Test_Double

Section outline

Stub

Fake

Mock

Test Spy

Dummy Object

Test Doubles

Racing Car Example

Alarm

+ check()

Sensor

+ sample_pressure()

Want to test this methodReplace with Test Double

Stub

Sensor

+ sample_pressure()

TestSensor

+ sample_pressure()

Same interfaceStub has no logic or advanced behaviour

a Stub is not the same as a Mock!

Test Doubles

Fake

File

+ seek()+ tell()+ readline()

StringIO


Same interface

Fake has logic and behaviour but is unsuitable for production

File


StringIO


Common things to replace with Fakes

icons: Double-J Design, FatCow

File Database WebServer

Test Doubles

Three kinds of Assert

Check the return value or an exception Check a state change (use a public API) Check a method call (use a mock or spy)

Increasing complexity

Interaction Testing

MyServiceTest

+ test_valid_token+ test_invalid_token

SSORegistry

+ register(id): token+ is_valid(token)+ unregister(token)No state change when

is_valid is called

MyService

+ handle(request, token)




MyService


MyServiceTest


SSORegistry

+ register(id): token+ is_valid(token)+ unregister(token)

TestDouble





Test Doubles

Test SpyMyService


SSORegistry


Spy


MyServiceTest


Test Spy

Spy


Mock


MyServiceTest


Fails the test straight away

Fails the test later on

Test Doubles

What’s the difference?

A Stub returns a hard coded answer to any query. It contains no logic. A Fake is a real implementation, but simpler - like a big complicated

Stub. A Mock is as a Stub, and additionally veri"es interactions. A Test Spy lets you query afterwards to "nd out what happened. A Dummy is something you use when the interface requires an

argument which isn't needed for the test.


Test Suite



Design the Units

Document the Units


Isolation

Test This one

This one is slow: replace with a Stub or Fake

Design the protocol to this one: use a Mock

Use the real one

Monkeypatching

Changing code at runtime!

Image by Martin Berube

Module Review

What is a Test Double? Stubs Fakes Mocks Spies Dummy Objects

Why use Test Doubles? isolation speed design

Monkeypatching to insert a test double

Unit Testing with PythonModule 6: Test Coverage


[email protected]





Module Overview

Parameterized tests with unittest and pytest

Measuring coverage of tests Using code coverage metrics when

adding test cases

Parameterized Tests

Same function or method Different parameters

Tennis

Love-All Fifteen-Love Fifteen-Thirty Fifteen-Forty

Icons by FatCow

Measuring Coverage

pip-3.3 install coverage pip-3.3 install pytest-cov

Gilded Rose Refactoring Kata

https://github.com/emilybache/GildedRose-Refactoring-Kata



Interpreting Coverage Data

Find missing test cases Get legacy code under test Continuous Integration - constant measurement

Coverage metric to aim for?

Test Quality != Coverage

Test Suite

Code Review

Bug Reports

Con!dence to Refactor

Flickering Tests

...

Module Review

Parameterized tests with unittest and pytest

Measuring coverage of tests to !nd missing test cases to improve regression protection

Interpreting coverage data

Software

Emily Bache - Unit Testing Python