Test Driven Development for Embedded C€¦ · Test Driven Development [email protected] Copyright ©2008-2010 James W. Grenning All Rights Reserved. For use by training

Test Driven Developmentwww.renaissancesoftware.net

[email protected] ©2008-2010 James W. GrenningAll Rights Reserved. For use by training attendees.

Presented at Agile China October 14, 2010

Test Driven Development for Embedded C

Presented by James Grenning

at Agile China 2010

twitter: jwgrenning

1

http://pragprog.com/titles/jgade/

1Monday, October 18, 2010

[email protected]

Copyright ©2008-2010 James W. GrenningAll Rights Reserved.

The Physics of Debug Later Programming (DLP)

• As Td increases, Tfind increases dramatically• Tfix is usually short, but can increase with Td

2

Bug discoveryMistake made(bug injection)

Bug found Bug fixed

Td Tfind T fix

Time


[email protected]

Copyright ©2008-2010 James W. GrenningAll Rights Reserved.

The Physics of Test Driven Development

• When Td approaches zero, Tfind approaches zero• In many cases, bugs are not around long enough to be considered

bugs.• See: http://www.renaissancesoftware.net/blog/archives/16

3

Mistake discovery

Mistake made

Root cause found

Mistake fixed

T d Tfind T fix

Time

3Monday, October 18, 2010Test Driven Development

[email protected]

Copyright ©2008-2010 James W. GrenningAll Rights Reserved. For use by training attendees.


TDD Micro Cycle

• Write a test• Watch it fail• Make it pass• Refactor (clean up any mess)• Repeat until done

4





TDD State Machine in C/C++

5

Write the test

Make the test link

Make the test compile

Compilation error

Link error

New test failsMake the test pass

Refactor(Make it right)

All tests pass

All tests pass No more tests

Choose a test

Start

Compilation error

Link error

DONE!

Programming error

Compiles Clean


[email protected]



Test Driving a Circular Buffer (FIFO)

6

23 7 66 12 99 16 90

Out-Index In-Index

99





Create a Test-List(avoid analysis paralysis)

7

Circular Buffer Tests

Initially Empty Transition to empty Transition to Full Transition from Full Put-Get FIFO Put to full Get from empty Filled but not wrapped around Wrap around


[email protected]



Set up a Test Fixture

8

TEST_GROUP(CircularBuffer){ CircularBuffer* buffer;

void setup() { buffer = CircularBuffer_Create(); }

void teardown() { CircularBuffer_Destroy(buffer); }};

TEST(CircularBuffer, TestName){}





Make a bare Skeleton of the Production Code

9

#ifndef D_CircularBuffer_H#define D_CircularBuffer_H

typedef struct CircularBuffer CircularBuffer;

CircularBuffer * CircularBuffer_Create(int capacity);void CircularBuffer_Destroy(CircularBuffer *);

#endif // D_CircularBuffer_H


[email protected]



Make a bare Skeleton of the Production Code

10

#include "CircularBuffer.h"#include <stdlib.h>

struct CircularBuffer{ int dummy;} ;

CircularBuffer* CircularBuffer_Create(){ CircularBuffer* self = calloc(capacity, sizeof(CircularBuffer)); return self;}

void CircularBuffer_Destroy(CircularBuffer* self){ free(self);}





Choose a Test

11

Write the test

Make the test link


Compilation error

Link error



All tests pass


Choose a test

Start

Compilation error

Link error

DONE!

Programming error

Compiles Clean


[email protected]



Which Test?

12

Out-Index In-Index

41 59 14 33 7 31

In-Index Out-Index





Production Code is Grown One Test at a Time

13

Code without

new feature

Code withnew tested

feature


[email protected]



Write a Test

14

Write the test

Make the test link


Compilation error

Link error



All tests pass


Choose a test

Start

Compilation error

Link error

DONE!

Programming error

Compiles Clean





Start with a Test That is Easy to Get to Pass -- Designing the Interface

15

TEST_GROUP(CircularBuffer){ CircularBuffer* buffer;

void setup() { buffer = CircularBuffer_Create(); }

void teardown() { CircularBuffer_Destroy(buffer); }};

TEST(CircularBuffer, ShouldBeEmptyAfterCreate){ CHECK(CircularBuffer_IsEmpty(buffer));}


Write the test

Make the test link


Compilation error

Link error



All tests pass


Choose a test

Start

Compilation error

Link error

DONE!

Programming error

Compiles Clean




Make the Test Compile

16





Make the Test Compile

17

#ifndef D_CircularBuffer_H#define D_CircularBuffer_H

typedef struct CircularBuffer CircularBuffer;

CircularBuffer * CircularBuffer_Create(int capacity);void CircularBuffer_Destroy(CircularBuffer *);int CircularBuffer_IsEmpty(CircularBuffer *);

#endif // D_CircularBuffer_H


Write the test

Make the test link


Compilation error

Link error



All tests pass


Choose a test

Start

Compilation error

Link error

DONE!

Programming error

Compiles Clean




Make the Test Link

18





Make the Test Linkand Intentionally Fail

19



... not all code shown ...

int CircularBuffer_IsEmpty(CircularBuffer* self){ return 0;}


[email protected]



Watch it Fail

20

CircularBufferTest.cpp:48: error: Failure in TEST(CircularBuffer, EmptyAfterCreation)

CHECK(CircularBuffer_IsEmpty(buffer)) failed


Write the test

Make the test link


Compilation error

Link error



All tests pass


Choose a test

Start

Compilation error

Link error

DONE!

Programming error

Compiles Clean




Make the Test Pass

21


[email protected]



Write no More Code than is Necessary to Pass the Current Test

• Hard code behavior or partial implementations are OK and often preferred.

22





Make the Test Pass

23



... not all code shown ...

int CircularBuffer_IsEmpty(CircularBuffer* self){ return 1;}


[email protected]



Repeat until You Run Out of Tests

• The first tests drive the interface definition• Later tests complete the behavior• You will think of more tests as you go

– Write the new test– or Add it to the test list

24





Tests are FIRST

Fast

Independent

Repeatable

Self-validating

Timely

25

Thank you: Tim Ottinger


[email protected]



Adaptation for Embedded Software

What is scarce during embedded development?

26





Hardware is Scarce!

• It does not exist.• It is being used by

someone else.•

27


[email protected]



Hardware is Scarce!

• It does not exist.• It is being used by

someone else.•

28


bool BlindsScheduler::DoesLightRespondToday(TimeService::Day reactionDay){ int today = timeService->GetDay();

if (reactionDay == TimeService::EVERYDAY) return true; if (reactionDay == today) return true; if (reactionDay == TimeService::WEEKEND && (TimeService::SATURDAY == today || TimeService::SUNDAY == today)) return true; if (reactionDay == TimeService::WEEKDAY && today >= TimeService::MONDAY && today <= TimeService::FRIDAY) return true; return false;}

void BlindsScheduler::CheckEvent(ScheduledBlindsEvent* blindsEvent){ if (blindsEvent->extraRandomEventPreventer > 0) { blindsEvent->extraRandomEventPreventer--; return; }

if (!DoesLightRespondToday(blindsEvent->day)) return; if (timeService->GetMinute() != blindsEvent->minute + blindsEvent->randomMinutes) return;

switch (blindsEvent->blindsType) { HorizontalBlinds* theBlinds; case Horizontal: switch (blindsEvent->operation) { case Raise:




Hardware Has Bugs of its Own

29


[email protected]



Tests Must be Fastand We Can Run Them NOW

30





Use Your Development Systemfor a Test Bed

• Multi-targeted code.• Must beware of hardware and OS dependencies.• Object Oriented approach to Dependency

Management.• Avoid inefficiencies

– Waiting for hardware.– Waiting for restart.– Waiting for downloads.– Waiting for long compiles.– Debugging on the target.

31


[email protected]



But There are Risks with Development System Tests

• Architecture differences– Word size– Big-endian Little-endian– Alignment

• Compiler differences• Library differences• Execution differences

32


Write a TestMake it Pass

Refactor

Stage 1

Compile for TargetProcessor

Stage 2

Run Tests in the Eval Hardware

Stage 3

Run Testsin Target Hardware

Stage 4

AcceptanceTests

Stage 5

More Frequent Less Frequent




TDD Adaptation for Embedded Development

33


[email protected]



TDD Adaptation for Embedded Development

34

See : http://renaissancesoftware.net/files/articles/ProgressBeforeHardware.pdf

Write a TestMake it Pass

Refactor

Stage 1

Compile for TargetProcessor

Stage 2

Run Tests in the Eval Hardware

or Simulator

Stage 3

Run Testsin Target Hardware

Stage 4

Run ManualTests in Target

Stage 5

More Frequent Less Frequent





Automate Stages 2-5

35

DeveloperWorkstation

Source CodeRepository

Continuous IntegrationServer

Target System

Eval/ReferenceSystem

Simulator

Test ManagementSystem


[email protected]



TDD and the Code in the Middle

(most of the code!)

36





What about Code in the Middle

The Module under testis tested independent of other modules

37


[email protected]



Testing a Module in the Middle

The test case takes the role of the client

The servers might need to be stubbed out

38





A Module with Collaborators

• Every minute, the RTOS wakes up the Light Scheduler.

• If it is time for one of the lights to be controlled, the LightController is told to turn on/off the light.

39

Time Service

+ getDay()+ getTimeOfDay()+ setPeriodicAlarm()

LightScheduler

+addSchedule()+removeSchedule()+wakeUp()

Light Controller

+ on(id)+ off(id)

Hardware RTOS

<<anonymous callback>>

Admin Console


[email protected]



Program to Interfaces

• Separate interface and implementation as separate entities.

• This design has good separation of responsibilities

40

<<interface>>

Time Service


LightScheduler


<<interface>>

Light Controller

+ on(id)+ off(id)

Hardware RTOS

<<anonymous callback>>

Real Light Controller

Real Time Service

<<implements>> <<implements>>

Admin Console





Testing a Module with Collaborators

• Use the real collaborators if you can.

• Use fakes when you must.

41

<<interface>>

Time Service


LightScheduler

Test

LightScheduler


<<interface>>

Light Controller

+ on(id)+ off(id)

Light Controller Spy

FakeTime Service

<<implements>> <<implements>>


[email protected]



Testing the Scheduler

42

TEST(LightScheduler, ScheduleOnTodayNotTimeYet){ LightScheduler_ScheduleTurnOn(3, EVERYDAY, 1000); FakeTimeSource_SetMinute(999);

LightScheduler_Wakeup();

LONGS_EQUAL(LIGHT_NA, FakeLightController_getState(3));}

TEST(LightScheduler, ScheduleOnTodayItsTime){ LightScheduler_ScheduleTurnOn(3, EVERYDAY, 1000); FakeTimeSource_SetMinute(1000);

LightScheduler_Wakeup();

LONGS_EQUAL(LIGHT_ON, FakeLightController_getState(3));}





Light Controller Interface

43

#ifndef D_LightController_H#define D_LightController_H

enum { MAX_LIGHTS = 32 };

void LightController_Create(void);void LightController_Destroy();void LightController_On(int id);void LightController_Off(int id);

#endif // D_LightController_H


[email protected]



Fake Light Controller Interface

44

#ifndef D_FakeLightController_H#define D_FakeLightController_H

#include "LightController.h"

typedef enum { NO_ID = -1, UNKNOWN_STATE = -1, LIGHT_OFF = 0, LIGHT_ON = 1} LightState;

int FakeLightController_getLastId(void);int FakeLightController_getLastState(void);int FakeLightController_getState(int id);

#endif // D_FakeLightController_H





Fake Light Controller Implementation

45

#include "FakeLightController.h"#include "memory.h"

static int lastId;static int lastLevel;

void LightController_Create(void){ lastId = NO_ID; lastLevel = UNKNOWN_STATE;}

void LightController_Destroy(){}


[email protected]



Fake Light Controller Implementation

46

int FakeLightController_getLastId(void) { return lastId;}

int FakeLightController_getLastState(void) { return lastLevel;}

void LightController_On(int id) { lastId = id; lastLevel = LIGHT_ON;}

void LightController_Off(int id) { lastId = id; lastLevel = LIGHT_OFF;}






47

TEST(LightScheduler, NoScheduleNothingHappens){ LightScheduler_Wakeup(); LONGS_EQUAL(NO_ID, FakeLightController_getLastId()); LONGS_EQUAL(UNKNOWN_STATE, FakeLightController_getLastState());}

TEST(LightScheduler, ScheduleOnTodayNotTimeYet){ LightScheduler_ScheduleTurnOn(3, EVERYDAY, 1200); FakeTimeService_SetMinute(1199); LightScheduler_Wakeup(); LONGS_EQUAL(NO_ID, FakeLightController_getLastId()); LONGS_EQUAL(UNKNOWN_STATE, FakeLightController_getLastState());}


[email protected]




48

TEST(LightScheduler, ScheduleWeekdayItsSunday){ LightScheduler_ScheduleTurnOn(3, EVERYDAY, 1200); FakeTimeService_SetDay(SUNDAY); FakeTimeService_SetMinute(1200); LightScheduler_Wakeup(); LONGS_EQUAL(3, FakeLightController_getLastId()); LONGS_EQUAL(LIGHT_ON, FakeLightController_getLastState());}





When must you use fakes?

• When the Code under test cannot be fully tested with the real collaborators.

• Examples:– When manual verification is needed (Printed output,

LEDs)– When the results change (Time, random events)– When failures need to be simulated (Network down)– When hardware is involved (LEDs, USB, Sensors,

Motors, IO pins, Flash...)– Operating system calls (RTOS)

49


[email protected]



Breaking Dependencies with Function Pointers

50





Loosening Compile-Time andRun-Time Dependencies

• Problem: an existing C function needs to be stubbed out for some tests...

51

//From the .h file...int RandomMinuteGenerator_Get();...

//From the .c file...int RandomMinuteGenerator_Get(){ ...the implementation... }...


[email protected]



Convert Direct Dependency to Function Pointer

• Modify the function declaration in the header file– Add extern– Add (*)

52

//From the .h file...extern void (*RandomMinuteGenerator_Get)();...





Convert Direct Dependency to Function Pointer

• Modify the function definition in the .c file– Change function name by appending _impl– Hide the _impl by making it static

• Add defining instance of the function pointer and initialize the pointer with the the _impl

53

//From the .c file...static int RandomMinuteGenerator_Get_impl(){ ...the implementation... }

int (*RandomMinuteGenerator_Get)() = RandomMinuteGenerator_Get_impl;


[email protected]



No Changes are Needed to Callers of the Function Pointer

• Calling code does not change. • No pointer dereferencing is needed

54

//from some production code... ... RandomMinuteGenerator_Get(); ...





Runtime SubstitutionTest setup and teardown

55

void setup(){ RandomMinuteGenerator_Get = RandomMinuteGenerator_Get_fake;}

void teardown(){ RandomMinuteGenerator_Get = RandomMinuteGenerator_Get_impl;}

• Override the function by assigning the test version of the function during setup

• Don’t forget to restore the original


[email protected]



Use CppUTest’s UT_PTR_SET

56

void setup(){ UT_PTR_SET(RandomMinuteGenerator_Get, RandomMinuteGenerator_Get_fake);}

void teardown(){}





Self Verifying Mock Objects

57


[email protected]



Message Flow for Flash Memory

Block Erase with Error

58

FlashDriver FlashDevice

IOWrite(CommandRegister, 0x40)

IOWrite(offset, data)

IORead(StatusRegister)

b7 == 0IORead(StatusRegister)

b7 == 1, other bits == 0

Repeats while b7 == 0

IORead(offset)

data

Flash_Program(offset, data)

FlashSuccess





Flash Program Flow Chart

59

How do you test these errors in the target?

How do you test these errors in the target?

How Many Tests are Needed?

b7 == 1

Start

Program CommandWrite 0x40 to 0x0

Start/Stop

Write data to address

Read status register

b3 == 0

b4 == 0

b1 == 0

YES

NO

YES

YES

YES

Vpp Error

Program Error

Protected Block Error

NO

NO

NO

Clear status Write 0xFF to 0x0


[email protected]



Mock Object

• Problem - Complex collaborator interactions cannot be captured with a simple spy.

• Solution - Mock Object– A Mock Object is a Test Double that verifies that the code

being tested interacts with its collaborator properly.– The test tells the mock

– The expected calls– In the expected order– What to return.

60





Flash Driver Test

61

TEST(Flash, ProgramSucceedsReadyImmediately1){ int result = 0; MockIO_Expect_IOWrite(0, 0x40); MockIO_Expect_IOWrite(0x1000, 0xBEEF); MockIO_Expect_IORead(0, 1<<7); MockIO_Expect_IORead(0x1000, 0xBEEF);

result = Flash_Program(0x1000, 0xBEEF);

LONGS_EQUAL(0, result); MockIO_Assert_AllExpectationsMet();}


[email protected]



Mock Flash Read/Write

62

//MockIO.h#include "IOReadWrite.h"

void MockIO_Create(int maxExpectations);void MockIO_Destroy();void MockIO_Expect_IOWrite(ioAddress_t offset, uint16_t data);void MockIO_Expect_IORead(ioAddress_t offset, uint16_t data);void MockIO_Assert_AllExpectationsMet();

//IOReadWrite.h#include <stdint.h>

typedef uint32_t ioAddress_t;typedef uint16_t ioData_t;

ioData_t IORead(ioAddress_t offset );void IOWrite(ioAddress_t offset, ioData_t data);





Mock Flash Read/Write

63

//Snipvoid MockIO_Create(int maxExpectations){ expectations = malloc(sizeof(Expectation)*maxExpectations); memset(expectations, 0, sizeof(expectations)); setExpectationCount = 0; getExpectationCount = 0; maxExpectationCount = maxExpectations; failureAlreadyReported = 0;}

//MockIO.ctypedef struct Expectation{ int kind; ioAddress_t addr; ioData_t value;} Expectation;

static Expectation* expectations = 0;static int setExpectationCount;static int getExpectationCount;static int maxExpectationCount;static int failureAlreadyReported = 0;


[email protected]



Capturing Flash Read/Write Expectations

64

void MockIO_Expect_IOWrite(ioAddress_t addr, ioAddress_t value){ failWhenNoMoreRoomForExpectations(report_too_many_write_expectations); recordExpectation(FLASH_WRITE, addr, value);}

void MockIO_Expect_IORead(ioAddress_t addr, ioAddress_t value){ failWhenNoMoreRoomForExpectations(report_too_many_read_expectations); recordExpectation(FLASH_READ, addr, value);}





Mock Flash Read

65

ioData_t IORead(ioAddress_t addr){ setExpectedAndActual(addr, -1); failWhenNotInitialized(); failWhenNoUnusedExpectations(report_read_but_out_of_expectations); failWhen(expectationIsNot(FLASH_READ), report_expect_write_was_read); failWhen(expectedAddressIsNot(addr), report_read_wrong_address);

return expectations[getExpectationCount++].value;}


[email protected]



Mock Flash Write

66

void IOWrite(ioAddress_t addr, ioData_t value){ setExpectedAndActual(addr, value); failWhenNotInitialized(); failWhenNoUnusedExpectations(report_write_but_out_of_expectations); failWhen(expectationIsNot(FLASH_WRITE), report_expect_read_was_write); failWhen(expectedAddressIsNot(addr), report_write_does_not_match); failWhen(expectedDataIsNot(value), report_write_does_not_match); getExpectationCount++;}





All Expectations Must Be Met

67

void MockIO_Assert_AllExpectationsMet(){ if (failureAlreadyReported) return; failWhenNotAllExpectationsUsed();}


[email protected]



Minimize DOH!

Debug

On

Hardware68





The End

69


[email protected]



On-line

• Test harnesses– [CPPTEST] www.sourceforge.org, project CppUTest– [FITNESSE] www.fitnesse.org

• Groups• http://groups.yahoo.com/group/testdrivendevelopment• http://groups.yahoo.com/group/AgileEmbedded

70





See Embedded TDD and Related Blogs and Papers

http://www.renaissanceSoftware.net http://www.renaissancesoftware.net/blog/

• Embedded TDD• Zune Bug: Test Driven Bug Fix• Learning Tests are Free! • TDD as a Design Rot Prevention System• Crashing Your Way to Great Legacy C Tests • TDD and the Big Framework Part• Bug Fixes and TDD• Physics of Test Driven Development• Tests vs. Short Term Cache Between Your Ears• Embedded Systems Conference FAQ • I miss constructors • Who says you can’t test drive a device driver?

71

• Why are You Still Using C? • Planing Poker• Agile Embedded Software Development (ESC) • Launching Extreme Programming at a Process

Intensive Company (IEEE) • Test Driven Development for Embedded Software• Progress Before Hardware • Agile Times - Containing Progress Before

Hardware• Test-Driven Development for Embedded C++

Programmers


[email protected]



References and Sources

• [RCM] Robert C. Martin, Clean Code, 2008• [SLAD] Craig Larman and Bas Voode, Scaling Lean & Agile

Development• [WELC] Michael Feathers, Working Effectively with Legacy Code• [REF] Martin Fowler, Refactoring• [TDD] Kent Beck, Test-Driven Development, 2003• [xUNIT] Gerard Meszaros - xUnit Testing Patterns, 2008• [MOCK] Tim Mackinnon and Steve Freeman and Philip Craig, Endo-

Testing: Unit Testing with Mock Objects, 2001• [TD] Lasse Koskela, Test Driven, 2007• [LESSONS] Bret Pettichord Cem Kaner, James Bach. Lessons Learned

in Software Testing: A Context-Driven Approach. John Wiley & Sons, New York, NY, 2002.

72





Available now in Beta

http://pragprog.com/titles/jgade/

In print early 2011

73


Documents

Test Driven Development for Embedded C€¦ · Test Driven Development [email protected] Copyright ©2008-2010 James W. Grenning All Rights Reserved. For use by training