Doing a Bradbury with Heckle

Andrew Grimm

University of New (South Wales)

@andrewjgrimm

http://andrewjgrimm.wordpress.com

What's Heckle?

Mutation tester

Makes changes to production code- mutations

Your unit tests should fail when changes are made – killing mutants

Heckle could be faster

Normally, each mutation requires the running of the specified test suite

Number of mutations proportional to size of codebase

Number of tests proportional to size of codebase

Therefore heckle time is ~ O(n^2)

Bradbury

Project name based after gold medal winning skater Steven Bradbury

Tells test/unit to stop running once each mutation has triggered a failing test

Should reduce running time

How test runs are stopped

Test::Unit allows you to add listeners triggered by updates, including failed tests

I've added a proc containing a request to throw a symbol

The calling of the test suite for a specific mutation occurs within a catch block

Future plans

Sort tests so that those likely to fail go first (the sooner you kill mutants, the sooner you can deal with the next one)

Random number seeding, to enable reproducible mutant creation

More future plans

Turn resistance to mutation into a percentage value

Add rcov-style code coloring

Color the method based on percentage resistance to mutation

Or highlight particular portions vulnerable to mutation

Make available to other code metric software

Rejected project names

Too hard to spell

(Jack) Kevorkian – euthanasia doctor

(Philip) Nitschke – euthanasia doctor

Too popular a name

Governator

Buster (from MythBusters)