Evolution of Longer-Term Planning in a Large Scale Agile Project – F-Secure’s experience

Protecting the irreplaceable | f-secure.com

Evolution of Longer-Term Planning in a Large Scale Agile Project – F-Secure’s experienceGabor Gunyho & Juan GutierrezImprovement Coach Manager, Agile Practices

2011-05-09

What is this presentation all about?

• Many publications in the

SW development literature

offer ready-made “recipes”

for solving a certain

problem

• We believe this is way too

often not helpful as the

context varies a lot

© F-Secure Public2011-05-092

Text source: http://easteuropeanfood.about.com/od/hungariansoups/r/gulyasleves.htm

What is this presentation all about?

• Instead we offer a story. This is how we did it in a big project.

• The story is told through the project's major events called “Release Planning”

• We hope that you‟ll learn some ideas on project planning and steering on a larger scale … even if it does not offer a ready-made recipe


Image source: http://lorabetht.blogspot.com/2010/11/violence-and-sophocles-antigone.html

About F-Secure


• Company

• Founded in 1988, listed on NASDAQ OMX Helsinki

• Market cap ca 350 m€, annual revenue ca 130 m€ (2010)

• Headquartered in Helsinki, 18 country offices, presence in more than 100 countries

• 850 people, 300+ in R&D, 5 R&D offices in 4 countries, Agile since 2003-2005

• Products and Services

• Online Security: Anti-Malware, e-mail filter, Browsing Protection, Parental Control

• Content Protection: Online Backup, Anti-Theft

• Online Storage and Services: Storage Platform, Sharing, Social Media Access

• Multiple OS platforms (Win, Mac, Linux, mobile), 20+ language versions

• Customers

• Consumers (retail, reseller, e-store), millions of homes

• Network operators (ISP, mobile), world leader with 200+ operator partners

• Corporate

About the Authors


Gabor Gunyho, Improvement Coach with the “R&D

Global Methods” team at F-Secure,

experienced Agile and Lean product

development expert, contributor and

reviewer of books on scaling Agile and

Lean SW development

Juan Gutierrez PlazaCurrently „Agile Practices Manager‟ at F-

Secure‟s Storage & Digital Content unit,

focusing on the R&D transformation of the site.

Experienced coach who has helped different

teams to improve in engineering and process

practices. Active member in different agile

forums and member of the board of Agile Spain

About the project

• Major new product, significant changes in

• Business model

• Architecture

• Method for Longer-Term Planning, including new backlog tooling

• About 10 teams

• Mostly in Helsinki, some in Kuala Lumpur, later also one in Poland

• Mostly feature teams

• Fairly mature in basic Scrum [1] and Agile engineering practices

• Some experience in multi-team projects [2] [3] but not on this scale


Longer-Term Planning: Why?

• Plan for developing a complex system with lots of unknowns

and lots of dependencies, both in features and in architecture

• Provide estimates and visibility to progress on a higher level of

abstraction of the content and timeline

• Handle the inherent organizational complexity in multi-team

setup

We need to get alignment in teams to a common

objective


“We need to improve the way how we manage our requirements, and especially

how we create concept (release) plans and link longer term architecture into our

short term plans“ Pirkka Palomäki, CTO

Longer-Term Planning in brief – the context

• Basic, Scrum-based Agile methodology does not cover scaling [1]

• Dean Leffingwell„s “Agile Release Train” [4] [5] covers multiple layers of

abstraction in all key dimensions of the project:

content, timeline and organization.


Product BacklogProduct

Owner 2Team B

I1 I2 I3 I4

Beta1

B

I

P

B

I

PI5 I6 I7 I8

Beta2

B

I

PI9 I10 I11 I12

Release

Epic Feature Story

Reporting Aggregate

Reports

As an user I want to see a list of my average

spending for each of my budget-lines so that I

can get a fast control of my average expenses

Reporting Aggregate

Reports

As a end-user I can get a summary report my

total spending on a selected set of accounts

Reporting List Report As a end-user I can get a summary report my


Reporting List Report As a end-user I can get a summary report my


Logging ...

Fea

ture

Sto

ryE

pic

Team A

Business Iteration

Product

Owner 1

BIP = Business Iteration Planning

PM

PM

AM

AM

Longer-Term Planning in brief – the event


Day 2

Status check

Planningteam breakout sessions

Final plan review

Risk review

Confidence vote

Retrospective

Day 1

Introduction

Project setup

Business Vision

Architecture Vision

User experience and UI

Engineering practices

Planning process intro

Planning team breakout sessions

Draft Plan review

The outcome: Plans vs. Planning

Plans are of little importance, but planning is essential

– Winston Churchill

Plans are nothing; planning is everything

– Dwight D. Eisenhower

No battle plan survives contact with the enemy

– Helmuth von Moltke the Elder

A good plan, violently executed now, is better than a

perfect plan next week

– George S. Patton


Source: http://en.wikipedia.org/wiki/Plan

http://en.wikipedia.org/wiki/Winston_Churchill

http://en.wikipedia.org/wiki/Dwight_D._Eisenhower

http://en.wikipedia.org/wiki/Helmuth_von_Moltke_the_Elder

http://en.wikipedia.org/wiki/George_S._Patton

The project journey – from the Longer-Term Planning perspective


New Method for the New Project


• New planning method: “Agile Release Train”

• Project kick-off and first “Release Planning” event: December 2009

• Planning Scope = 90 days

• 6 sprints, 2 weeks each, plus next planning

• Program for the "release” planning event

• One day training on Scaling SW agility

• “Release” Planning: 2 days +1 day reserve (that had to be used after all)

• Attendants: ~120, including all functions (R&D, Product Mgmt,

business, etc)

• Everybody (almost) in one space

• Facilitator: Dean Leffingwell, supported by internal improvement

coaches

Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar2010 2011

Biz Iteration #1P

Learning from the Business Iteration #1

• Beta releases: 2 internal (though delayed), 1 failed

• Huge amount of content becomes apparent, many dependencies and risks

• Feature vs. story concept is not clear to many

• Planning event can be done in 2 days (keep a 3rd day in reserve though)

• The 90-day “Business Iteration” with mid-term re-planning is not meaningful

• Change the cadence to 8-week Business Iterations with proper planning for each

• Changes in the planning process:

• Internal event facilitator takes over from external guru

• Moving towards continuous “Flow” in planning e.g., “Draft plan review” scrapped, instead, synchronize planning work of teams in every hour

• Special-skill experts (business, architecture, UX, etc) available throughout the event



Biz Iteration #1P P

1 2 3

Planning event #2 (End of March 2010)

• Preceded by a ½ day retrospective done with Open Space [6]

• Slow feedback in development

• Improve Build System, Test Automation and reduce release effort

• “Release” Planning fixes: focus on dependencies, limit sprint load to team‟s

capacity

• Architecture challenges: different levels of abstraction needed, establish

virtual team for architects

• Changes for “Release” Planning event #2

• Imbalance between scope and velocity recognized and

acknowledged

• Change project setup:

Full system release June 2010 ->

Variant-A release by Aug (+2 months), Variant-B by Dec (+ 6 months)

• Plan for releasing a beta (for customer preview) in every sprint



Biz Iteration #1 Biz Itrtn #2P P

1 2 3

Planning event #3 (early June 2010)

• Beta releases: 2 of 4 successful

“Based on R&D analysis of project scope and schedule

it is apparent that the current roadmap is not feasible”

Project split

• Spin off “mini-project” for downscaled Variant-A

• Variant-B development continues, with staged delivery,

Variant-B basic release Oct 2010, full set release Jan 2011

• Changes in the planning process

• At every hour synchronization meetings alternate,

one for the planning work and another for architecture issues

• Helpers for planning the improvement items rotate in teams

• Risk, impediment and dependency handling in short cycles (along with the

synch meetings), and not at the end of the planning event

• “Feature wall “ (or “master wall”) to depict which feature will be done by

whom and when, also visualizing dependencies



Biz Iteration #1 Biz Itrtn #2 Biz Iteration #3P P P

54 6 71 2 3

Example of a Master Wall

• Master wall that shows which team works on which

feature in which iteration

• List of identified risks and impediments

• ROAMed - Resolved, Owned, Accepted, Mitigated



Biz Iteration #1 Biz Itrtn #2 Biz Iteration #3P P P

54 6 71 2 3

Planning event #4 (early Sept 2010)


• Full-time ScrumMasters for each team

• Event delayed by 1 week, time used for

• hardening (bug killing)

• re-teaming event: moving towards more feature teams

• ScrumMaster training


• Separate intro briefing session, including solution demo

• All input in physical tokens (all features printed)

• Improved master wall, physical visualization of dependencies

• Clear priorities for planning: “honesty over precision”

• Smoother planning in general



Biz Iteration #1 Biz Itrtn #2 Biz Iteration #3 Biz Iteration #4P P P P

54 6 7 13 14 15 16 17 181 2 3

Planning event #4 - memories



Biz Iteration #1 Biz Itrtn #2 Biz Iteration #3 Biz Iteration #4P P P P

54 6 7 13 14 15 16 17 181 2 3

Planning event #5 (early Dec 2010)


• 1st public beta release

• “Last mile” before release

• Company restructuring

• Feature leaks (slipping content)

Last-minute change in the planning process

“We believe the last mile before release is better executed with feature focused planning.”


• New internal event facilitator

• New “feature planning” (see next)

• More emphasis in backlog grooming within sprints and Product Backlog preparation by Product Owners

• Release when last good build is deemed by Product Owners as good to go

Business Iteration (BI) Planning method is not abandoned, it’s just not applied for the last mile of the project


BI #4

BI #3

BI #2

BI #1

Planned

Done

Nr of

features


Biz Iteration #1 Biz Itrtn #2 Biz Iteration #3 Biz Iteration #4 “Last mile”P P P P P

54 6 7 13 14 15 16 17 18 19 20 21 221 2 3 18.5 23

Feature Planning for the “last mile” before release


Team

X

Team

Y

Develop

Feature A

Develop

Feature D

I(n) I(n+1) I(n+2) I(n+3) I(n+4) I(n+5)

Develop

Feature B

Develop

Feature C

Beta(n) Beta(n+1)

Develop

Feature E Backlog grooming

is about:- User needs and

requirements

- Architecture

- Dependencies

- UX

- Risks

of the next featureDevelop

Feature F

Features prepared

or refined by

Product Owners

in Product Backlog

regularly

Grooming for Feature D Grooming for Feature E

Grooming for Feature C Grooming for Feature F

Product

Owners,

etc

Iterations

Beta(n+2) Beta(n+3) Beta(n+4) Beta(n+5) Beta releases

(public)



54 6 7 13 14 15 16 17 18 19 20 21 221 2 3 18.5 23

Towards the end of the project

• As of mid-Feb, 2011 for the first time the burndown shows that the remaining

work can be done by the planned release date


Note: data on picture was adjusted for work in progress items not reflected on picture

when the above conclusion was made



54 6 7 13 14 15 16 17 18 19 20 21 221 2 3 18.5 23 24 2625 27 28

… and at the end of the project

• As of March 30, 2011 CR1 (Commercial Release #1) was released




54 6 7 13 14 15 16 17 18 19 20 21 221 2 3 18.5 23 24 2625 27 28 29 30 31

Summary of the method

• New method for handling layers of abstraction in all key dimensions

• Business Iteration for steering in mid-term time scale

• Levels of abstraction in the “Value item” hierarchy: epics, features, stories

• Planning for the Business Iteration with the features and stories in a multi-

team setting

• Essential to pay attention to quality and the engineering practices like

Continuous Integration and Test Automation

• Never sacrifice quality, never

• Every bug found invokes adding a new test case to the Test Automation

suite

• No extra hardening outside of sprints, every sprint results in a customer

beta


Conclusions - the morale of the story

Better visibility and steerability for business

management,

helping making tough decisions

• Inspect and adapt, ie, evolve the method as you go

• Get real feedback after almost every sprint, even if product is big

• Get help from an experienced “process master”, ie, event facilitator

• Prepare the content well and do it continuously, don‟t overload the system

• Synchronization meetings within the planning, later extended to continuous

handling of risks, dependencies and impediments

• Presence of all stakeholders (business, architects, user experience)

• Master wall and feature coverage tracking

• Move to “feature planning”, a precursor to continuous planning


References

[1] Schwaber, K., Beedle, M.: Agile Software Development with Scrum. Prentice Hall (2001)

[2] Larman, C., Vodde, B.: Scaling Lean & Agile Development: Thinking and Organizational Tools for

Large-Scale Scrum. Addison-Wesley Professional (2008)

[3] Larman, C., Vodde, B.: Practices for Scaling Lean & Agile Development: Large, Multisite, and

Offshore Product Development with Large-Scale Scrum. Addison-Wesley Professional (2010)

[4] Leffingwell, D.: Scaling Software Agility: Best Practices for Large Enterprises. Addison-Wesley

Professional (2007)

[5] Leffingwell, D.: Agile Software Requirements: Lean Requirements Practices for Teams, Programs,

and the Enterprise. Addison-Wesley Professional (2011)

[6] http://en.wikipedia.org/wiki/Open_Space_Technology


Questions?


Contact [email protected]@f-secure.com

This presentation, and the project behind it are the work of many people whom all we can‟t possibly list here.

We‟d like to thank all who made this possible, most notable the whole team of this project.

Special thanks to F-Secure‟s R&D Management for supporting the work and this presentation

The authors did their best to attribute the authors of texts and images, and to recognize any copyrights, see more

details of copyrights, license terms and conditions for each source under the reference link provided. If you think

that anything in this material should be changed, added or removed, please contact the authors at the addresses

above


http://creativecommons.org/licenses/by-nd/3.0/

mailto:[email protected]









Background slides(actual fact sheets)

2011-05-0929 © F-Secure Public

Schedule look before planning event #1.5 (re-plan)


Week Iteration Comments

50 7.12. Iteration 1

51 18.12.

52 21.12. Iteration 2

53 1.1.

1 4.1. Iteration 3

2 15.1.

3 18.1. Iteration 4 PSI-1 re-planning End-to-end testable

internal (LAB) release4 29.1. beta-1

5 1.2. Iteration 5

6 12.2. beta-1 (12.2.2010)

7 15.2. Iteration 6

8 26.2. beta-2 (26.2.2010)

9 1.3. Iteration 7

10 12.3. beta-3 & PSI-1

(12.3.2010)

11 15.3. Release-retrospective & PSI2 planning

12 19.3. PSI-2 iterations begin…

Schedule

100%

25

%

30.6.

7.12.

29.1.

∼60% of

PSI-1

PSI = Potentially Shippable Increment, earlier term for Business Iteration

Scoping before planning #2

• Number of features before planning #2

2011-05-0931 © F-Secure Public

Nice to have

Important

Mandatory

Variant-A

Common

Variant-B

Schedule and scope adjustment before planning #2


Var-A

Var-B

RTM

Var-A

RTM

Var-B

RTM

Original

Current approved

+2 months +6 months

PSI planning

PSI planning

PSI planning

PSI planning

PSI planning

PSI-1 PSI-2 PSI-3 PSI-4 PSI-5 PSI-6

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan

Planning

for this now

2010

PSI = Potentially Shippable Increment, earlier term for Business Iteration

RTM = Release to Manufacturing

2011

Trade-off matrix for Variant-A as of planning #2


PERFORMANCE of the back-end Improve Keep Sacrifice

PERFORMANCE of the client Improve Keep Sacrifice

USER EXPERIENCE Improve Keep Sacrifice

USABILITY Improve Keep Sacrifice

BUSINESS support for F-Secure Improve Keep Sacrifice

SECURITY Improve Keep Sacrifice

FEATURE SET of the client Improve Keep Sacrifice

FEATURE SET of the back-end Improve Keep Sacrifice

INTEROPERABILITY Improve Keep Sacrifice

RELIABILITY Improve Keep Sacrifice

SUPPORTABILITY Improve Keep Sacrifice

TESTABILITY in R&D Improve Keep Sacrifice

Legend: Comparing to our client & backend offer of August 2009

• IMPROVE this area

• KEEP this area roughly equally good

• SACRIFICE things in this area to succeed in others

Trade-off matrix for Variant-B as of planning #3


PERFORMANCE of the client Improve Keep Sacrifice

USER EXPERIENCE Improve Keep Sacrifice

USABILITY Improve Keep Sacrifice

BUSINESS support for F-Secure Improve Keep Sacrifice

SECURITY Improve Keep Sacrifice

FEATURE SET of the client Improve Keep Sacrifice

FEATURE SET of the back-end Improve Keep Sacrifice

INTEROPERABILITY Improve Keep Sacrifice

RELIABILITY Improve Keep Sacrifice

SUPPORTABILITY Improve Keep Sacrifice

TESTABILITY in R&D Improve Keep Sacrifice

Legend: Comparing to our client & backend offer of August 2009

• IMPROVE this area

• KEEP this area roughly equally good

• SACRIFICE things in this area to succeed in others