Present Development Test-Driven Development Test in (bed ...Test-Driven Development in (bed) with C by James W Grenning - [email protected] Bas Vodde - [email protected] Presented

Test-Driven Development in (bed) with C

by James W Grenning - [email protected]

Bas Vodde - [email protected]

www.odd-e.comwww.renaissancesoftware.net

Presented at Agile 2011, Salt Lake City, UT, USA

and

Present

Scaling Lean & Agile Development

Thinking and Organizational Tools for Large-Scale Scrum

Craig LarmanBas Vodde

Practices for Scaling Lean & Agile

DevelopmentLarge, Multisite, and Offshore Products

with Large-Scale Scrum

Craig LarmanBas Vodde

[email protected]

Copyright © 2008-2011 James W. Grenning, Bas VoddeAll Rights Reserved. .

This Work Flow is Designed to Allow Defects

2

Development

Test

Defects

[email protected]


Edsger DijkstraThose who want really reliable software will discover that they must find means of avoiding the majority of bugs to start with, and as a result, the programming process will become cheaper. If you want more effective programmers, you will discover that they should not waste their time debugging, they should not introduce the bugs to start with.

3 [email protected]


Can we Realize Dijkstra’s Dream andPrevent Defects with

Test Driven Development?

4

[email protected]


Development and Test Work TogetherPreventing Defects

5

Development

Test

[email protected]


The Physics of Debug Later Programming (DLP)

• As Td increases, Tfind increases dramatically• Tfix is usually short, but can increase with Td

6

Bug discoveryMistake made(bug injection)

Bug found Bug fixed

Td Tfind T fix

Time

[email protected]


The Physics of Test Driven Development

• When Td approaches zero, Tfind approaches zero• In many cases, bugs are not around long enough to be considered

bugs.• See: http://www.renaissancesoftware.net/blog/archives/16

7

Mistake discovery

Mistake made

Root cause found

Mistake fixed

T d Tfind T fix

Time

[email protected]


What is TDD?

8

[email protected]


TDD Micro Cycle

• Write a test• Watch it not compile• Make it compile, but fail• Make it pass• Refactor (clean up any mess)• Repeat until done

9

The one rule: Only ever write code to fix a failing testwww.renaissancesoftware.net

[email protected] © 2008-2011 James W. Grenning, Bas VoddeAll Rights Reserved. .

TDD State Machine in C/C++

10

Write the test

Make the test link

Make the test compile

Compilation error

Link error

New test failsMake the test pass

Refactor(Make it right)

All tests pass

All tests pass No more tests

Choose a test

Start

Compilation error

Link error

DONE!

Programming error

Compiles Clean

[email protected]


Test Driving a Circular Buffer (FIFO)

11

Out-Index In-Index

23 7 66 12 99 16 90

Out-Index In-Index

42 -6 23 7 66 12 99 16 90

Out-Index In-Index

99

[email protected]


Create a Test-List(though, avoid analysis paralysis)

12

Circular Buffer Tests

Initially Empty Transition to empty Transition to Full Transition from Full Put-Get FIFO Put to full Get from empty Filled but not wrapped around Wrap around

[email protected]


Think about the Solution(though, avoid analysis paralysis)

• We’ll need two indexes, one for putting integers in and one for taking them out

• We’ll need an array, that is dynamically sized during creation

• We’ll need wrap around logic

13 [email protected]


Set up a Test Fixture

14

TEST_GROUP(CircularBuffer){ CircularBuffer* buffer;

void setup() { buffer = CircularBuffer_Create(); }

void teardown() { CircularBuffer_Destroy(buffer); }};

TEST(CircularBuffer, TestName){}

[email protected]


Make a bare Skeleton of the Production Code

15

#ifndef D_CircularBuffer_H#define D_CircularBuffer_H

typedef struct CircularBuffer CircularBuffer;

CircularBuffer * CircularBuffer_Create();void CircularBuffer_Destroy(CircularBuffer *);

#endif // D_CircularBuffer_H

[email protected]


Make a bare Skeleton of the Production Code

16

#include "CircularBuffer.h"#include <stdlib.h>

struct CircularBuffer{ int dummy;} ;

CircularBuffer* CircularBuffer_Create(){ CircularBuffer* self = calloc(1, sizeof(CircularBuffer)); return self;}

void CircularBuffer_Destroy(CircularBuffer* self){ free(self);}

[email protected]


Choose a Test

17

Write the test

Make the test link


Compilation error

Link error



All tests pass


Choose a test

Start

Compilation error

Link error

DONE!

Programming error

Compiles Clean

[email protected]


Which Test?

18

Out-Index In-Index

41 59 14 33 7 31

In-Index Out-Index

[email protected]


Production Code is Grown One Test at a Time

19

Code without

new feature

Code withnew tested

feature

[email protected]


Write a Test

20

Write the test

Make the test link


Compilation error

Link error



All tests pass


Choose a test

Start

Compilation error

Link error

DONE!

Programming error

Compiles Clean

[email protected]


Start with a Test That is Easy to Get to Pass -- Designing the Interface

21

TEST_GROUP(CircularBuffer){ CircularBuffer* buffer;

void setup() { buffer = CircularBuffer_Create(); }

void teardown() { CircularBuffer_Destroy(buffer); }};

TEST(CircularBuffer, ShouldBeEmptyAfterCreate){ CHECK(CircularBuffer_IsEmpty(buffer));}

Write the test

Make the test link


Compilation error

Link error



All tests pass


Choose a test

Start

Compilation error

Link error

DONE!

Programming error

Compiles Clean

[email protected]


Make the Test Compile

22

[email protected]


Make the Test Compile

23

#ifndef D_CircularBuffer_H#define D_CircularBuffer_H

typedef struct CircularBuffer CircularBuffer;

CircularBuffer * CircularBuffer_Create();void CircularBuffer_Destroy(CircularBuffer *);int CircularBuffer_IsEmpty(CircularBuffer *);

#endif // D_CircularBuffer_H

Write the test

Make the test link


Compilation error

Link error



All tests pass


Choose a test

Start

Compilation error

Link error

DONE!

Programming error

Compiles Clean

[email protected]


Make the Test Link

24

[email protected]


Make the Test Linkand Intentionally Fail

25



... not all code shown ...

int CircularBuffer_IsEmpty(CircularBuffer* self){ return 0;}

[email protected]


Watch it Fail

26

CircularBufferTest.cpp:48: error: Failure in TEST(CircularBuffer, ShouldBeEmptyAfterCreate)

CHECK(CircularBuffer_IsEmpty(buffer)) failed

Write the test

Make the test link


Compilation error

Link error



All tests pass


Choose a test

Start

Compilation error

Link error

DONE!

Programming error

Compiles Clean

[email protected]


Make the Test Pass



Write no More Code than is Necessary to Pass the Current Test

• Hard code behavior or partial implementations are OK and often preferred.

28

[email protected]


Make the Test Pass

29



... not all code shown ...

int CircularBuffer_IsEmpty(CircularBuffer* self){ return 1;}

[email protected]


Repeat until You Run Out of Tests

• The first tests drive the interface definition• Later tests complete the behavior• You will think of more tests as you go

– Write the new test– or Add it to the test list

30

Code without

new feature

Code withnew tested

feature

[email protected]


Exercise



Post-Exercise Discussion

32

[email protected]


But that is so Inefficient!So Slow!

Show us the fast way!



Get good at being careful and you can go fast.

34

The Careful way is the Fast Way!

[email protected]


Why is it difficult to sustain?

35

TDD:• Steady pace• Speed limits• No traffic lights (bugs)• Might feel slow!!

Traditional:• Spurts• Fast when no problems!• Debugging• Feels fast! But often slow

[email protected]


Sustainability

36

Speculate Code Test DebugTraditional development

Time developers do not notice nor

plan for

Time vs

Test-driven development

Speculate Test and Code Debug

Feels slower

[email protected]


37

Stuff you need to know!

[email protected]


Tests Must Be Automated

38

[email protected]


39

It is not a unit test when:• It talks to the database• It communicates across the network• It touches the file system• It can't run at the same time

as any of your other unit tests• You have to do special things to your

environment (such as editing config files) to run it.

Michael Featherswww.renaissancesoftware.net

[email protected] © 2008-2011 James W. Grenning, Bas VoddeAll Rights Reserved. .

Refactoring

40

Structured code transformation to prepare the code

for change

Key points:• Doesn’t adjust functionality• Small and disciplined• Well defined• Keeps code healthy

[email protected]


Code Smells

4133

A sign there is probably something wrong with your code

It smells!

It stinks!

[email protected]


Modularity == Testability

42

Focus on tests first ensures modularity and other good design principles

It’s about design, stupid!

[email protected]


43

Design from the perspective of userather than implementation

[email protected]


CppUTest Cheat Sheet

44

/* in CheatSheetTest.cpp */#include "CppUTest/TestHarness.h"

/* Declare TestGroup with name CheatSheet */TEST_GROUP(CheatSheet) {/* declare a setup method for the test group. Optional. */ void setup () {/* Set method real_one to stub. Automatically restore in teardown */ UT_PTR_SET(real_one, stub); }

/* Declare a teardown method for the test group. Optional */ void teardown() { }}; /* Do not forget semicolumn */

/* Declare one test within the test group */TEST(CheatSheet, TestName){ /* Check two longs are equal */ LONGS_EQUAL(1, 1);

/* Check a condition */ CHECK(true == true);

/* Check a string */ STRCMP_EQUAL("HelloWorld", "HelloWorld");}

/* In allTest.cpp */IMPORT_TEST_GROUP(CheatSheet);

/* In main.cpp */

#include "CppUTest/CommandLineTestRunner.h"#include "AllTests.h"

int main(int ac, char** av){ return CommandLineTestRunner::RunAllTests(ac, av);}

[email protected]


CppUTest Mocking Cheat Sheet

45

/* Additional include from CppUTestExt */#include "CppUTest/TestHarness.h"#include "CppUTestExt/MockSupport.h"

TEST_GROUP(MockCheatSheet){ void teardown() { /* Check expectations. Alternatively use MockSupportPlugin */ mock().checkExpectations(); }};

TEST(MockCheatSheet, foo){ /* Record 2 calls to Foo. Return different values on each call */ mock().expectOneCall("Foo") .withParameter("param_string", "value_string") .withParameter("param_int", 10) .andReturnValue(30); mock().expectOneCall("Foo") .ignoreOtherParameters() .andReturnValue(50);

/* Call production code */ productionCodeFooCalls();}

TEST(MockCheatSheet, bar){ /* Expect 2 calls on Bar. Check only one parameter */ mock().expectNCalls(2, "Bar") .withParameter("param_double", 1.5) .ignoreOtherParameters();

/* And the production code call */ productionCodeBarCalls();}

/* Stubbed out product code using linker, function pointer, or overriding */int foo(const char* param_string, int param_int){ /* Tell CppUTest Mocking what we mock. Also return recorded value */ return mock().actualCall("Foo") .withParameter("param_string", param_string) .withParameter("param_int", param_int) .returnValue().getIntValue();}

void bar(double param_double, const char* param_string){ mock().actualCall("Bar") .withParameter("param_double", param_double) .withParameter("param_string", param_string);}

/* Production code calls to the methods we stubbed */void productionCodeFooCalls(){ int return_value; return_value = foo("value_string", 10); return_value = foo("value_string", 10);}

void productionCodeBarCalls(){ bar(1.5, "more"); bar(1.5, "more");}

[email protected]


Embedded

46

[email protected]


Use C or C++?

• Why C++ (e.g. gcc):

• Able to use C++ unit test framework

• Able to use C++ features in tests

• Why C:

• Not annoyed by the small differences

• Able to use a C compiler.

• E.g. run tests in “real environment”



Hardware is Scarce!

• It does not exist.• It is being used by

someone else.• It has bugs of its

own.

48

[email protected]


The Real Thing?

• Run unit tests on real hardware?

• Probably not. Too slow.

• Every now and then it could be possible.

• Use the real compiler?

• Periodically. With check in.

• Do run higher level tests on real HW every now and then!

[email protected]


But There are Risks with Development System Tests

• Architecture differences– Word size– Big-endian Little-endian– Alignment

• Compiler differences• Library differences• Execution differences

50

Write a TestMake it Pass

Refactor

Stage 1

Compile for TargetProcessor

Stage 2

Run Tests in the Eval Hardware

Stage 3

Run Testsin Target Hardware

Stage 4

AcceptanceTests

Stage 5

More Frequent Less Frequent

[email protected]


TDD Adaptation for Embedded Development



TDD Adaptation for Embedded Development

52

See : http://renaissancesoftware.net/files/articles/ProgressBeforeHardware.pdf

Write a TestMake it Pass

Refactor

Stage 1

Compile for TargetProcessor

Stage 2

Run Tests in the Eval Hardware

Stage 3

Run Testsin Target Hardware

Stage 4

AcceptanceTests

Stage 5

More Frequent Less Frequent

[email protected]


Problems Found at Appropriate Stage

53

Stage Problems Likely to Find

1 Logic, design, modularity, interface, boundary conditions

2 Compiler compatibility (language features)Library compatibility (header files, prototypes)

3 Processor executions problems (bugs in compiler and standard libraries)Portability problems (word size, alignment, endian)

4 Ditto stage 3Hardware integration problemsMisunderstood hardware specifications

5 Ditto stage 4Misunderstood feature specification

[email protected]


How do I test Code with Hardware and OS Dependencies?

54

[email protected]


A Module with Collaborators

• Every minute, the RTOS wakes up the Light Scheduler.

• If it is time for one of the lights to be controlled, the LightController is told to turn on/off the light.

55

Time Service

+ GetTime()+ SetPeriodicAlarm()

LightScheduler

+ScheduleTurnOn()+RemoveSchedule()+WakeUp()

Light Controller

+ On(id)+ Off(id)

Hardware RTOS

<<anonymous callback>>

Admin Console

[email protected]


Program to Interfaces

• Separate interface and implementation as separate entities.

• This design has good separation of responsibilities

56

<<interface>>Time Service


LightScheduler

+ ScheduleTurnOn()+ RemoveSchedule()+WakeUp()

<<interface>>Light Controller

+ On(id)+ Off(id)

Model 42 Hardware RTOS

<<anonymous callback>>

Model 42Light Controller

RTOS Time Service

<<implements>> <<implements>>

Admin Console

[email protected]


Testing a Module with Collaborators

• Use the real collaborators if you can.

• Use fakes when you must.

57

<<interface>>Time Service


LightScheduler

Test

LightScheduler

+ ScheduleTurnOn()+ RemoveSchedule()+wakeUp()

<<interface>>Light Controller

+ On(id)+ Off(id)

Light Controller Spy

FakeTime Service

<<implements>> <<implements>>

[email protected]


Testing the Scheduler

58

TEST(LightScheduler, ScheduleOnTodayNotTimeYet){ LightScheduler_ScheduleTurnOn(3, EVERYDAY, 1000); FakeTimeSource_SetMinute(999);

LightScheduler_Wakeup();

LONGS_EQUAL(LIGHT_NA, LightControllerSpy_GetState(3));}

TEST(LightScheduler, ScheduleOnTodayItsTime){ LightScheduler_ScheduleTurnOn(3, EVERYDAY, 1000); FakeTimeSource_SetMinute(1000);

LightScheduler_Wakeup();

LONGS_EQUAL(LIGHT_ON, LightControllerSpy_GetState(3));}

[email protected]

Copyright © 2008-2011 James W. Grenning, Bas VoddeAll Rights Reserved. TDD with Collaborating Objects

Partial Skeleton Let’s You Try the Design and Test Ideas Early


Copyright © 2008-2011 James W. Grenning, Bas VoddeAll Rights Reserved. TDD with Collaborating Objects

f2fp refactoring

void function (int para); void function (int para){ do_implementation ();}

Header Source

before

afterextern void (*function) (int para);

void function_imp (int para){ do_implementation ();}

void (*function)(int para) = function_imp;

[email protected]


Stubbing

• Stub level:

• Preprocessor (rare)

• Function pointer

• Link

• Stub type

• Exploding

• Sensible default

• Recording

• Generic



Level: preprocessor

#include "stubs.h"

void something (){ function(100);}

#define function(a) function_stub(a, b)

Source Stub header

Advantages: Disadvantages:

- Creates lots of flexibility.

-> for example. Can stub out just one call.

- Changes the production code

- Requires different build configurations

[email protected]


Level: function pointer


- Ability to runtime change

- Pretty safe

- Requires f2fp refactoring

- One extra call

- Dangling function points (avoided when using UT_PTR_SET)

TEST_GROUP(group){ void setup () { UT_PTR_SET(real_one, stub); }

[email protected]


Level: link stubs


- Can give a lot of flexibility!

- No change in production code

- Easy to re-use stubs

- Be careful of different configurations. Just have one stub (use generic link stubs)

- Difficult (impossible) to call “the real thing”’

- Don’t use on file-level. Otherwise it causes many executables and stubs.

void function (){ /* Do stubbed things */}

[email protected]


Type: Sensible default

int function (){ return 10;}

Useful when stubbing out 3rd party libraries

[email protected]


Type: Exploding

void function (){ FAIL("stub: function was called");}

Especially useful when starting. When it explodes something went wrong or you need to implement it.

[email protected]


Type: Dynamic

Benefits of both function pointer and link level stubs!

void (*function_stub) () = NULL;

void function (){ if (function_stub) function_stub ();}

[email protected]


Type: Recording (mock)

Very generic usage. int function (int parameter){ mock().actualCall("function") .withParameter("parameter", parameter) .returnValue().getIntValue();}

[email protected]


Type: Combination

int function (int parameter){ if (function_stub) return function_stub(parameter); mock().actualCall("function") .withParameter("parameter", parameter);

if (mock().hasReturnValue()) return mock().returnValue().getIntValue(); return 0;}

Very flexible.

[email protected]


Closing Thoughts

70

[email protected]


Why is it Worth Changing How You Program

• Defects kill predictability:–Cost of fixing is not predictable–When they materialize is not predictable

• Test-driven is predictable:–Working at a steady pace–Results in fewer bugs–More productive than “debug-later programming”

• Test-driven programmers rarely need the debugger



TDD is Defect Prevention

72

Test and FixTest and Fix

Test and Fix

Test and Fix


Test and Fix

Test and Fix



Test and Fix


CAN PREVENT CODE FIRES

Test and Fix

Test and Fix

Test and Fix

[email protected]


Questions

73

Documents

Present Development Test-Driven Development Test in (bed ...Test-Driven Development in (bed) with C by James W Grenning - [email protected] Bas Vodde - [email protected] Presented