ESC-404 Boston 2010 James W. Grenning... [email protected] Copyright ©2008-2010 James W. Grenning, All Rights Reserved A Bright Idea! 7

[email protected]

Copyright ©2008-2010James W. Grenning, All Rights Reserved

Agile Requirements, Estimation and Planning --

Iteration ZeroESC-404 Boston 2010

James W. [email protected]

twitter: jwgrenning

1 [email protected]


Agile Principles

• Communications• Simplicity• Feedback• Courage• Respect

• Visibility• Honesty• Realistic• High Quality

2

[email protected]


Agile Practices• Incremental and iterative development• Adaptive Planning • Test as we go• Automated Test• Test Driven Development

3 [email protected]


Why Iterative?• A system's users seldom know exactly what

they what and cannot articulate all they know• … There are many details we can only

discover once we are well into implementation• … as humans we can only master only so

much complexity• … external forces lead to changes in

requirements…[LARMEN]

4

[email protected]


Track Projects Based on Functionality Built and Tested

Requirements

Design

Code

Test

5

Com

plet

e St

orie

s

Sprints or Iterations (2-4 weeks)

Lear

ning

[email protected]


Get Ready to Iterate- Iteration Zero -

6

[email protected]


A Bright Idea!

7 [email protected]


What does the Product Do?

8

The product must

• Secure password

• Schedule lights

• Schedule blinds

• Security camera

• Perimeter alarm

• Phone police

• Call brother-in-law

• Entry delay

• Party mode

• Software updates

• Network config

The product must

• Secure password

• Schedule lights

• Schedule blinds

• Security camera

• Perimeter alarm

• Phone police


• Entry delay

• Party mode


• Network config

The product must

• Secure password

• Schedule lights

• Schedule blinds

• Security camera

• Perimeter alarm

• Phone police


• Entry delay

• Party mode


• Network config

[email protected]


Critical Dates• Internal releases• Trade shows• Version 1 release• Version 2 release

9

November

Sun Mon Tues Wed Thurs Fri Sat

[email protected]


Iteration Zero Participants• People with a vision of the product being

developed• People that understand why features are

needed and how they will be used• People that will build the system• People that will test the system• People that fund the system• Technology and Domain Experts

10

[email protected]


Iteration Zero

The product must

• Secure password

• Schedule lights

• Schedule blinds

• Security camera

• Perimeter alarm

• Phone police


• Entry delay

• Party mode


• Network config

The product must

• Secure password

• Schedule lights

• Schedule blinds

• Security camera

• Perimeter alarm

• Phone police


• Entry delay

• Party mode


• Network config

The product must

• Secure password

• Schedule lights

• Schedule blinds

• Security camera

• Perimeter alarm

• Phone police


• Entry delay

• Party mode


• Network config

November


Close Blinds Action

Add(id, Action, day, minute)Remove(id, Action, day, minute)Wake Up()

Scheduler

Open Blinds Action

Turn Off Light Action

Turn On Light Action

<<implements>>

Execute()

<<interface>Action

<<interface>>Time Service

+ GetTime()+ SetPeriodicAlarm()

<<interface>>Light Controller

+ On(id)+ Off(id)

Model 42 Hardware

RTOS

Model 42Light Controller

RTOS Time Service

<<implements>>

<<implements>>

Admin Console

BlindsScheduler

LightScheduler

BlindsOnDemand

LightsOnDemand

E-Mail

TExt Message Siren

Twitter

<<implements>>

Notify()

<<interface>Notifier

Alarm Detection

The Bright Idea

Critical Dates

Product Needs

InitialArchitectural Vision

operatorWorkstation

XCB NCS

Status Panel

A/D Collector

Backend DB

TSC

Team Practices Agreement

TDDStoriesFitnesseStand up2 week iterationsCode review per story

Estimated Story Backlogand Release Plan

Technology Goals

Team members identified

Product Vision



Inputs and Outputs

[email protected]


Architectural Vision- Not Big Formal Documents -

- Guiding Ideas -

12

Close Blinds Action


Scheduler

Open Blinds Action



<<implements>>

Execute()

<<interface>Action




+ On(id)+ Off(id)

Model 42 Hardware

RTOS


RTOS Time Service

<<implements>>

<<implements>>

Admin Console

BlindsScheduler

LightScheduler

BlindsOnDemand

LightsOnDemand

E-Mail

TExt Message Siren

Twitter

<<implements>>

Notify()


Alarm Detection

operatorWorkstation

XCB NCS

Status Panel

A/D Collector

Backend DB

TSC

Expect the vision to change and evolve with learning

[email protected]


The Team and its Sponsors• Who is on the team?• What roles are missing?• Who are the sponsors?

13 [email protected]


Team Agreement• Self-organizing team• You own the process• How will you work• Expect to evolve your

practices• Decide how to decide• Decide to be a team

14



[email protected]


After Iteration 0...Start Iterating

15

The near term planis made up of important workin small pieces, vertical slicesof functionality.

The long term planis a mix of small and largergrained features.

Iteration Zero

[email protected]


Make it Visible- Use Big Visible Charts -

16

Close Blinds Action


Scheduler

Open Blinds Action



<<implements>>

Execute()

<<interface>Action




+ On(id)+ Off(id)

Model 42 Hardware

RTOS


RTOS Time Service

<<implements>>

<<implements>>

Admin Console

BlindsScheduler

LightScheduler

BlindsOnDemand

LightsOnDemand

E-Mail

TExt Message Siren

Twitter

<<implements>>

Notify()


Alarm Detection

operatorWorkstation

XCB NCS

Status Panel

A/D Collector

Backend DB

TSC

November


The product must

• Secure password

• Schedule lights

• Schedule blinds

• Security camera

• Perimeter alarm

• Phone police


• Entry delay

• Party mode


• Network config

The product must

• Secure password

• Schedule lights

• Schedule blinds

• Security camera

• Perimeter alarm

• Phone police


• Entry delay

• Party mode


• Network config

The product must

• Secure password

• Schedule lights

• Schedule blinds

• Security camera

• Perimeter alarm

• Phone police


• Entry delay

• Party mode


• Network config

DevelopersJeff, Pete, Mary, Suresh, James, Rainu, Phil, Gary

Customer TeamDennis, Thierry, Gary, Lou, Cathy

SponsorsJohn, Victor

[email protected]


Stakeholders and

Customer Team



Whole Team

18

Devel

PO

Tester

Devel

Devel

Devel

Devel

DevelTester

Tester

BA SE

stories and tests

working software

Customerteam

Developerteam

[email protected]


Key Stakeholders and Customer Team

• Who are the people that–define what is needed?–that develop the system?–that test the system?–that fund the product?–manufacture the system?

• Every organization is unique, each development effort is unique



Fine Grained Scope Control with User Stories

20

[email protected]


Introducing the User Story• The name of a feature. • A promise for a conversation. (Ron Jeffries)

• Like the name of a use case, or extension.–Acceptance tests provide the details.

• Fine grains help make visible progress and avoid gold plating.

• I call them Product Stories

21

Weekend Light Schedule

Weekday Light Schedule

Specific day Light Schedule

Chinese Holiday Light Schedule

US Holiday Light Schedule

Everyday-but Light Schedule

Everyday Light Schedule

[email protected]


Stories and Acceptance Tests

• Stories lack detail• Details are provided in automated

acceptance tests• The test are like executable use cases• Test either pass or fail

22

[email protected]


High Level Requirements to Stories

23

Focusedstories

!/$ !/$

!/$ !/$ !/$ !/$!/$

!/$

!/$!/$

!/$

!/$

!/$

!/$!/$

!/$

!/$

Deliver the most bang

for the buck first.

The product must

• Secure password

• Schedule lights

• Schedule blinds

• Security camera

• Perimeter alarm

• Phone police


• Entry delay

• Party mode


• Network config

The product must

• Secure password

• Schedule lights

• Schedule blinds

• Security camera

• Perimeter alarm

• Phone police


• Entry delay

• Party mode


• Network config

The product must

• Secure password

• Schedule lights

• Schedule blinds

• Security camera

• Perimeter alarm

• Phone police


• Entry delay

• Party mode


• Network config

Scheduling Notification SecurityEpic stories (story areas)

[email protected]


Stories Allow MuSCoW Analysis• Maximize Value delivered by date using

MuSCoW–Mu - Must have–S - Should have–Co - Could have–W - Won’t have

24

[email protected]


Canonical Form of a Story

As a [role]I want [function]

so that [business value].



My Preference• Big visible feature name written so you

can read it from a distance.• Optional role and business value

–But always be prepared to add them if asked.

26

Name of Function

[role]

[business value]

[email protected]


Back of the Card• Optionally, use the back of the card for

details, and notes about acceptance criteria.

• Keeping the specification light until more detail is needed JIT.

27

Schedule a light to turn on a specific day at 8PM

At the scheduled time, on the scheduled day - light is turned on- otherwise it is unchanged

Specific day Light Schedule

[email protected]


Invest in User StoriesSource: Mike Cohn

Independent

Negotiable

Valuable

Estimate-able

Small

Testable

28

[email protected]


INVEST• The INVEST model can be a challenge for

embedded.(its a challenge for non-embedded too)



Independent• Challenging because of Hardware, OS

and other dependencies.• Fake it ‘til you make it.(Kent Beck)

• Design up to an interface, then fake it.• Find important work that can be made

independent.

• Another use of I -- Immediately actionable.

30

[email protected]


Negotiable• When a story is too big, break it down.• It’s too small, aggregate.• Split known and from unknown.• Extract a spike from the unknown.• Split by tests.• Split by what can be shown first in the

natural order of development.



Valuable• The ideal story delivers value to the

customer.• May be unrealistic in for some embedded

stories. • Sometimes Visible is all you can get.

32

VValuable

Visible Progressor

[email protected]


Estimable• Supports planning and tracking.• Inhibitors to estimation:

–Domain knowledge.–Technical knowledge.–Invention

• what is really needed?–Too big.



Small• Stories must be small enough to be

completed in an iteration.• You think that is small...

–Multiple stories should be completed in each iteration

• Story Weight Reduction Toolkit - 4 S’s–Split, Spike, Stub, Time box– http://www.renaissancesoftware.net/blog/archives/48

34

[email protected]


Testable• Tests define “done”• Tests are requirements.• Tests clarify the need, the solution.• Tests are determined just in time.• Test definition happens upstream, not at

the bottom of the waterfall.

35

One big story, with greater risk and no feedhback

Four smaller stories, with less risk and more feedhback

Value realized

[email protected]


Why Small and Visible?

36

[email protected]


What if Part of the Work is not Valuable or Urgent?

One big story, with greater risk and no feedhback

Four smaller stories, with less risk and more feedhback

Value realizedSooner

Work that can be avoided

or delayed



Example Epic Story• Save configuration data in the flash file

system–Domain issues

• What configuration data?• How is it organized? How is it recalled?• Batch or on-demand?

–Technical issues• How to build a flash driver?• How to build a file system?• What file system standard?

–Size issues• Cannot deliver story in an iteration?

38

[email protected]


Try to find a natural order• USB device detected• Device identified as a flash drive• Erase flash memory device• Open an existing file• Read from an existing file• Open a file for writing• Write to the file



Consider the Customer

40

Turn On flash LED

Hardware engineer

To verify address logic

Who benefits from the story

Why is it valuable

[email protected]


Stories are a Vertical Slice of Functionality

• They cut across subsystem boundaries• They can include

–UI (graphics, preliminary or polished)–System behavior

• They have an agreed upon definition of DONE.

• Prefer visible work over engineering tasks



The Backlog is Made up of Stories

• The short term plan is more detailed.• Work on it, buying time to refine longer term plan.

• Generally stories are in the order set by customer.• Engineers can ask to move up stories to reduce

risk.• Stories are tested in the iteration they are

implemented; story tests are automated.• A story is done when it passes its tests.

42

[email protected]


Grow the SystemDemonstrated with Working

Requirements

Design

Code

Test

43

Com

plet

e St

orie

s

Sprints or Iterations (2-4 weeks)

Lear

ning

[email protected]


Stories Pull in Supporting Work Products

• Stories pull in work from other groups.• You could annotate a card (e.g. add a red

dot) to indicate other group’s work is needed.

• A story in one of the next couple iterations, gives visibility of the need for certain supporting assets

44

[email protected]


Visibility and Just in Time

45

Imagine that artwork needs to be integrated with software. Plan and deliver JIT

[email protected]


Teamwork

First Order Effect Great Software is built by great teams of people

46

[email protected]


Collaboration• Daily standup meeting• Pair programming or Daily reviews• Shared code ownership• Team room



Agile Estimation and Planning

48

[email protected]


Project Cost and Schedule Uncertainty

49

time

1x

2x

4x

.25x

.5x

Project Complete

Cos

t U

ncer

tain

ty

InitialDefinition

RequirementsSpecification

Design

1x

1.25x

1.6x

.6x

.8x

Sch

edul

e U

ncer

tain

ty

Barry Boehm, 1995

[email protected]


Agile Lifecycle• Potential to release on any iteration

boundary

• In transition from Waterfall to Agile, some teams precede releases with regression tests and hardening

50

Iteration Zero

Trade Show V1.0 V1.1

[email protected]


Estimation and Planning1.First determine relative sizes of stories in

Story Points.2.Estimate team velocity (story points per

iteration).3.Lay out a Release Plan.4.Calibrate plan.5.Feedback into the plan with measured

velocity.6.Regularly revise the plan.



Wisdom of the Crowds[by Daren Brown]

• Teams do better than experts.• Diversity within a group is needed.• The more diverse the knowledge and

opinions of the group, the smarter the group.

• A random group does better than an expert group.Ask the audience? (reportedly, Philbin once said that the audience's answer is statistically 95% of the time correct.)See blog article:http://www.renaissancesoftware.net/blog/archives/20

52

[email protected]


Bulk Story Sizing --- Planning Poker Party

• High-Low Showdown• Deal and Slide• Developer Guts• Customer Guts

• Read more about it here– http://www.renaissancesoftware.net/blog/archives/36



Affinity Grouping• Group by similar effort• Pick one of the easier stacks of similar

value to be a “1”• Use high-med-low stacks where there are

many stories, then use affinity on the low stack.

54

[email protected]


Before High-Low Showdown



After High-Low Showdown

56

[email protected]


Deal and Slide



Assign Relative Effort to Each Column

58

[email protected]


Developer Guts• Estimate (guess) team velocity

Velocity = points completed in one iteration



VelocityV = story points completed per iteration

• Initially estimated.• Later measured as estimated points

completed

• Never dictated or “stretched”.• Never compared between groups.

60

[email protected]


Measure Development VelocityEstimated work per Iteration

61

0

5

10

15

20

25

30

35

40

45

50

1 2 3 4 5 6 7 8 9 10

Velo

city

in S

tory

Poi

nts

Iterationwww.renaissancesoftware.net

[email protected] ©2008-2010

James W. Grenning, All Rights Reserved

Product Burn Down ChartWork to be Completed

62

0

75

150

225

300

1 2 3 4 5 6 7 8 9 10

Sto

ry P

oint

s R

emai

ning

Iteration

[email protected]


Customer Guts• Lay out the release plan as a series of

iterations.• Total story points per iteration cannot exceed

estimated velocity.• Near-term iterations

usually are higher valueor risk.

• Further out plan is more vague, less certain.



Change Becomes Visible

64

0

75

150

225

300

1 2 3 4 5 6 7 8 9 10

New features added--what

happens?

[email protected]


Plan to Replan• The plan is wrong, its an educated guess.• Replan every few iterations, or as needed• Do another Planning Poker Party• When small batches of stories are brought

in by the customer, use Planning Poker



Planning Poker• Popular estimation technique• Not as useful for large batches of stories

–Prefer Planning Poker Party• Good for estimating small batches of

stories where a baseline already exists• Engaging for the whole team• It’s fun

Read more about it http://renaissancesoftware.net/papers/44-planing-poker.html

66

[email protected]


Planning Poker Deck



Planning Poker Mechanics• Each player has a hand of planning poker

cards.• Customer reads a story.• Until estimates converge

–Developers discuss to make sure they understand the story, not how they would build it.

–Each secretly chooses their estimate.–All flip their card simultaneously.–Discuss extremes, re-deal.

68

[email protected]


Team Agreement on Practices

Doing it!



Adoption Recommendations1.Two week iterations2.Daily standup3.Collaborate4.Track Visually5.Break work into User Stories6.Define acceptance test as you go7.TDD new code

-Add tests to legacy code as you go

8.Continuously integration70

[email protected]


Tracking Recommendations• Management

–Velocity• Should be stable, but not constant

–Total Story Points in Backlog–Backlog burn down chart–Backlog growth (feature creep, bug stories)

• Quality–Number of bugs that escape the iteration

• Process–Number of tests per iteration (AT and UT)



Discuss and Decide

• Iteration length• Iterations meeting day

and time• Daily standup meeting

time• TDD• Automated Acceptance

Testing • Continuous integration• Coding standard

• Pair programming• Code reviews• Shared code ownership• Shared workspace/lab• Tracking

–Race track–Effort burn down

• Retrospectives• Other?

72

[email protected]


Transition Backlog• Learning goals?• Tools research and setup?• Organizational changes?• Other?

• Setup a transition team• Get some help



Non-Planning Iteration Zero Activities

• Training in Agile Development and Test Driven Development

• Setup tools for Test Driven Development• Setup tools for Story Testing (Acceptance

testing)• Setup a continuous integration server

(Hudson, Cruise Control, for example)• Schedule meetings• Adjust office

74

[email protected]


Iteration Planningand Working in Iterations



Iteration Planning Meeting• Reviews the previous iteration

–Demo of completed stories–Retrospective

• What worked? What did not work? What should we do differently?

• Plan next iteration–Review stories, tests, acceptance criteria–Check for over/under commitment–Design discussions and task breakdowns

76

[email protected]


In Each Iteration• Scope is fixed, later iterations are open to

change.• Developer team use evolutionary design

techniques.• Customer team (Business and QA Test)

– is on call– elaborates acceptance tests– researches future iteration’s stories

• Re-plan periodically• Use Planning Poker for small batches of stories

added to the backlog http://renaissancesoftware.net/papers.html



The Work in an Iteration• Keep stories small• Break bigger stories into tasks, or split

stories• Developers choose their own work• Daily stand up

78

[email protected]


Iteration Tracking• Story Race Course• Half the story points delivered and tested

half the way through• Daily stand up



Race Track

80

Selected Started Dev Done Accepted

[email protected]


Race Track

81

Selected Started Dev Done Accepted

[email protected]


Daily Stand Up Meeting• Attendees: whole team• When: every day, at the same time, on-

time, with whoever shows up

• Three points... quickly–What I did yesterday, what I am doing today,

what is in my way.• Further discussion between interested

parties after the standup

82

[email protected]


Adoption Challenges



Change is Disruptive

84

Pe

rfo

rma

nce

Time

1. Late Status Quo

2. Resistance

3. Chaos

4. Integration

5. New Status Quo

ForeignElement

Transforming Idea

Virginia Satir Change Model

[email protected]


Success Factors and Critical Skills• Management support and involvement• Realize its not just an engineering problem• Get help to reduce the chaos period• Champion(s) needed

• Walk the walk - Talk the talk–Manage with feedback and data–Stay flexible, don’t commit too early–Teamwork –Evolutionary Design and Continuous Testing



Agile Practices• Support each other• Align with the values• Are not an end in them selves• Each team is unique

86

[email protected]


Challenges• Legacy code• Automated testing• Stories• Culture• Getting out of the cube and into the team• Engineers get too specialized



88

http://agilemanifesto.org/

[email protected]


89

http://www.halfarsedagilemanifesto.org/

[email protected]


Getting Started• Motivation for change• Open to different approaches• Learn• Experiment• TDD under the radar• Stories for individual work• Management support• Coaching helps

90

[email protected]


Embedded Developers Biggest Concerns

(Jack Ganssle quotes Embedded Forecasts study)

• Poor Requirements• “Resources” a.k.a. People• Schedule

AGILE Provides a more realistic approach to managing each of those concerns.



Coming SoonFall 2010

92

[email protected]


See Related Blogs and Papers

http://www.renaissanceSoftware.net http://www.renaissancesoftware.net/blog/

• Embedded TDD• Zune Bug: Test Driven Bug Fix

• Learning Tests are Free! • TDD as a Design Rot Prevention System

• Crashing Your Way to Great Legacy C Tests • TDD and the Big Framework Part

• Bug Fixes and TDD• Physics of Test Driven Development

• Tests vs. Short Term Cache Between Your Ears• Embedded Systems Conference FAQ

• I miss constructors

• Who says you can’t test drive a device driver?

93

• Why are You Still Using C?

• Planing Poker

• Agile Embedded Software Development (ESC)

• Launching Extreme Programming at a Process Intensive Company (IEEE)

• Test Driven Development for Embedded Software

• Progress Before Hardware

• Agile Times - Containing Progress Before Hardware

• Test-Driven Development for Embedded C++ Programmers

[email protected]


References and Resources• [SLAD] Craig Larman and Bas Voode, Scaling Lean & Agile Development• [POP] Mary Poppendieck and Tom Poppendieck, Implementing Lean

Software Development: From Concept to Cash, 2006• [AGILE] Robert C. Martin, Agile Software Development: Principles,

Patterns, and Practices, 2002• [CLEAN] Robert C. Martin, Clean Code, 2008• [TDD] Kent Beck, Test-Driven Development, 2003• [XP] Kent Beck, Extreme Programming Explained, 1999• [REF] Martin Fowler. Refactoring. Improving the Design of Existing Code.

1999• [WELC] Michael Feathers, Working Effectively with Legacy Code• [XUNIT] Gerard Meszaros, xUnit Testing Patterns, 2008• [PRAG] Andy Hunt, Dave Thomas, The Pragmatic Programmer• [KANER] Cem Kaner, et. al. Lessons learned in Software Testing• Lasse Koskela, Test Driven, 2007

94

[email protected]


On-line

• Test harnesses–[CPPTEST] www.sourceforge.org, project CppUTest

–[FITNESSE] www.fitnesse.org• Groups• http://groups.yahoo.com/group/testdrivendevelopment• http://groups.yahoo.com/group/AgileEmbedded



Questions and Discussion

96