Better Testing Through Behaviour

Better Testing Through BehaviourOpen Source Developers’ Conference

November 2007

Tom AdamsWorkingmouse

“The act of writing a unit test is more an act of design than of verification. It’s also more an act of documentation than of verification. The act of writing a unit test closes a remarkable number of feedback loops, the least of which is the one pertaining to verification of function.”

Robert C. Martin

Why do I care?I. Better process

•Workflow is easier, IDE-a-bility, simpler

•Supporting infrastructure & language that guides you down the correct path

II. Better results

•Splitting of state into separate contexts means understanding

•Nicer syntax, less code to mentally parse

•Readable results

Test driven development•Development practice, arising out of XP’s test first approach

•Incremental process - test, code, refactor, test code, refactor…

•Drives implementation - improved quality, low coupling, high cohesion

•Comprehensive regression test suite

•Focus on design?

•Focus on documentation?

•Focus on behaviour?

Problems?•Too much time/money, optional extra

•Overlapping with traditional QA testers

•Developers require training

•Developer resistance

•Vocabulary difference

Problems?•Test vocabulary affects thinking

•Intent not clear

•Design focus obscured

•Documentation

•1-1 mapping of test/production code

•Tight tests coupled to code

•Repetitive infrastructure setup

•Where to start?

BDD is just TDD•Coined by Dan North - JBehave first BDD framework

•Development practice, arising out agile methodologies

•Refinement of TDD that shifts the emphasis from testing to specification

•Frameworks - JBehave, RSpec, Instinct, JDave, NSpec, GSpec, beanSpec, JSSpec, DSpec, MissingBDD, PHPSpec, Specs, Specipy

class ACsvFileReaderWithNothingToRead { @Subject CsvFileReader csvFileReader; @Mock CsvFile csvFile; @Stub CsvLine[] noLines;

@Specification void returnsNoLines() { expect.that(new Expectations() {{ one(csvFile).hasMoreLines(); will(returnValue(false)); ignoring(csvFile).close(); }}); expect.that(csvFileReader.readLines()).equalTo(noLines); }}

class AnEmptyStack { @Subject Stack<Object> stack; @Dummy Object object;

@Specification void isEmpty() { expect.that(stack.isEmpty()).isTrue(); }

@Specification void isNoLongerBeEmptyAfterPush() { stack.push(object); expect.that(stack.isEmpty()).isFalse(); }

@Specification(expectedException = IllegalStateException.class, withMessage = "Cannot pop an empty stack") void throwsExceptionWhenPopped() { stack.pop(); }}

AnEmptyStack - isEmpty - isNoLongerBeEmptyAfterPush - throwsExceptionWhenPoppedANonEmptyStack - isNotEmpty - isNoLongerFullAfterPoppingAllElements - throwsExceptionWhenANullIsPushed - popsPushedValue - shouldPopSecondPushedValueFirst - leavesValueOnStackAfterPeek

Ubiquitous language•Language has an impact on how you think about something

•Known in linguistics as the Sapir-Whorf hypothesis

•There is “a systematic relationship between the grammatical categories of the language a person speaks and how that person both understands the world and behaves in it”

•The language used to describe software constructs has an impact on how we create those constructs, e.g. good APIs, well named variables

Ubiquitous language•Getting the words right - naming of classes, methods and variables

•Borrows from domain driven development (DDD)

•Bridges gap between technical and business artefacts

•Captures the behaviour of the domain using clear and concise syntax

•Forms consensus around domain artefacts and run-time behaviour

private void testRunnerSendsSpecifiationResultsToOutput() { assertTrue("Expected to find context name", runnerOutput.contains(className));}

private void sendsSpecifiationResultsToOutput() { expect.that(runnerOutput).containsString(className);}

assertEquals(1, map.size());assertTrue(map.containsKey(1000));assertEquals(fileNames, map.get(1000));

expect.that(map).hasSize(1);expect.that(map).containsKey(1000);expect.that(map).containsEntry(1000, fileNames);

public void testMethodIsAnnotated() { checkIsAnnotated(WithRuntimeAnnotations.class, Context.class, true);  checkIsAnnotated(WithoutRuntimeAnnotations.class, Context.class, false);}

private void checkIsAnnotated(AnnotatedElement element, Class<?> expectedAnnotation, boolean expectingAnnotation) { AnnotationChecker annotationChecker = new AnnotationCheckerImpl();  assertEquals(expectingAnnotation, annotationChecker.isAnnotated(element, expectedAnnotation));}

public void testClassIsAnnotated() { expect.that(annotationChecker.isAnnotated( WithRuntimeAnnotations.class, Context.class)).isTrue(); expect.that(annotationChecker.isAnnotated( WithoutRuntimeAnnotations.class, Context.class)).isFalse();}

Design focus•Design is one of the most important aspect of TDD

•TDD’d code is (usually of) higher quality that non-TDD’d code

•Bugs, coupling, cohesion, maintainability, understandable, smaller, etc.

•Emphasis on testing limits TDD’s uptake & effectiveness

•Organisational, technical, process

•Encourages you to think about design

•Design is documented through ubiquitous language, contexts

Behaviour focus•Focused specifications

•One context per “state” of subject

•Minimal expectations per specification method

Behaviour focus•1-to-1 test-to-prod code mapping broken

•“Units” gone, what’s important is the behaviour

•Specification code is less coupled to production code

•Easier refactoring

•M-to-N mapping encouraged

class AnEmptyStack { void isEmpty() {} void isNoLongerBeEmptyAfterPush() {} void throwsExceptionWhenPopped() {}}

class ANonEmptyStack { void isNotEmpty() {} void throwsExceptionWhenANullIsPushed() {} void popsPushedValue() {} void shouldPopSecondPushedValueFirst() {} void leavesValueOnStackAfterPeek() {}}

Process•Works best top-down/outside-in

•Specify at the highest level first

•Use top-level objects to discover the services needed from the next level down

•Rinse, repeat

Levels•Story-level

•Code-level

•Mostly historical, frameworks are now adopting both approaches

•Both levels are legitimate, use one or the other depending on audience

Scenario "transfer from savings account to cheque acount" do Given "my savings account balance is", 100 do |balance| @savings_account = Accounts::AccountFactory.create(:savings) @savings_account.add(balance) end And "my cheque account balance is", 50 do |balance| @cheque_account = Accounts::AccountFactory.create(:cheque) @cheque_account.add(balance) end When "I transfer", 20 do |amount| @savings_account.transfer(amount.to_i).to(@cheque_account) end Then "my savings account balance should be", 80 do |balance| @savings_account.balance.should == balance end And "my cheque account balance should be", 70 do |balance| @cheque_account.balance.should == balance endend

describe "non-empty Stack" do it "should return the top item when sent #peek" do @stack.peek.should == @last_item_added endend

class ANonEmptyStack { void shouldReturnTheTopItemWhenSentPeek() { expect.that(stack.peek()).equalTo(lastItemAdded); }}

Instinct•Goals - explicitness, simplicity and flexibility

•Code-level (currently) framework

•Unified state and behaviour (mocking) expectation API (c.f. xUnit Assert)

•Built in infrastructure - mocks, stubs, dummies, subjects

•Formalised nomenclature

•Integration - JUnit, Ant, Clover, IntelliJ IDEA

Instinct examples

Summary•Shifts the emphasis from testing to specification

•Provides a ubiquitous language

•Strong focus on design

•Emphasises system behaviour, independent of where the behaviour resides

Take homeI. Better process

•Workflow is easier, IDE-a-bility, simpler

•Supporting infrastructure & language that guides you down the correct path

II. Better results

•Splitting of state into separate contexts means understanding

•Nicer syntax, less code to mentally parse

•Readable results

References•http://en.wikipedia.org/wiki/Behavior_driven_development

•http://blog.daveastels.com/2005/07/05/a-new-look-at-test-driven-development

•http://blog.davidchelimsky.net/files/BDDWithRspec.RubyConf.2007.pdf

•http://code.google.com/p/instinct/

•http://rspec.rubyforge.org/