SEA - Challenges In Scaling Scrum First...••Scrum can significantly improve developer job satisfaction, but that is a secondary result. Business objectives are driving. ••Scrum

4/4/2013

1

Challenges In Scaling ScrumChallenges In Scaling Scrum

Robert WardRobert Ward3 April 20133 April 2013

The Agile Manifesto In ContextThe Agile Manifesto In Context

•• The Manifesto is mostly heuristics, not mandates and The Manifesto is mostly heuristics, not mandates and

not first principles.not first principles.

•• It aimed to legitimize resistance to “conventional It aimed to legitimize resistance to “conventional

wisdom” about software development. wisdom” about software development.

•• The manifesto was what the diverse signatories could The manifesto was what the diverse signatories could

agree to at the time. agree to at the time.

•• We can now say much more about how and why Agile We can now say much more about how and why Agile

works.works.

Copyright © 2013, Robert Ward

4/4/2013

2

Serving Business ObjectivesServing Business Objectives

•• PredictabilityPredictability

•• Adaptability/Risk ManagementAdaptability/Risk Management

•• Return on InvestmentReturn on Investment

•• Customer SatisfactionCustomer Satisfaction

Agile is management refactored

for product development.


Agile Builds OnAgile Builds On

•• Probability Theory and SPCProbability Theory and SPC

•• High Performance Teams (HPT) and what we High Performance Teams (HPT) and what we know about meaningful work. know about meaningful work.

•• Lean ManufacturingLean Manufacturing and Theory of Constraintsand Theory of Constraints

•• Queuing TheoryQueuing Theory

•• Continuous improvement techniques from TQMContinuous improvement techniques from TQM

•• Decision TheoryDecision Theory

•• Experimental research into creativity and Experimental research into creativity and programming. programming.


4/4/2013

3

Agile First PrinciplesAgile First Principles

•• Software Development is always affected by Software Development is always affected by many unknowable random variables. many unknowable random variables.

•• Stable random processes are Stable random processes are very very predictable in the aggregate.predictable in the aggregate.

•• Agile reporting and planning exploits SPC. Each Agile reporting and planning exploits SPC. Each Velocity measurement is a sample from a Velocity measurement is a sample from a continuous, stable process. Scrum measures continuous, stable process. Scrum measures process output in estimated scope units (story process output in estimated scope units (story points), absorbing both random variation in the points), absorbing both random variation in the estimation and in the production time. estimation and in the production time.


Conflicting Ideas You Conflicting Ideas You Might Might HearHear

•• With planning, sprint review, and retrospective, With planning, sprint review, and retrospective, two week sprints incur too much overhead. We two week sprints incur too much overhead. We need at least four weeks.need at least four weeks.

•• There’s no point in reThere’s no point in re--estimating every day. All we estimating every day. All we need to track is which stories get finished.need to track is which stories get finished.

•• We should just plan a three week “release sprint” We should just plan a three week “release sprint” for all the things we end up needing to do at the for all the things we end up needing to do at the end of the release cycle. end of the release cycle.


4/4/2013

4

Traditional ManagementTraditional Management

Maximize Utilization RatesMaximize Utilization Rates

Random

Arrivals

Deterministic

Service Time

Production

Resource



•• Units of Software change are not Units of Software change are not

produced in deterministic time (Software produced in deterministic time (Software

pipelines are pipelines are MMnMMn, NOT , NOT MDnMDn, systems). , systems).

•• Loading an Loading an MMnMMn system at more than system at more than

70% will create serious bottlenecks. 70% will create serious bottlenecks.

From

Queueing

Theory


4/4/2013

5

MDnMDn vs. vs. MMnMMn systemssystems

Queue Size vs. Capacity Utilization

Qu

eu

e S

ize

Percent Capacity UtilizationCopyright © 2013, Robert Ward

Agile/Lean ManagementAgile/Lean Management

Maximize Flow RatesMaximize Flow Rates

•• Monitor Queues To Identify BottlenecksMonitor Queues To Identify Bottlenecks

•• CrossCross--train and Swarmtrain and Swarm

•• Queues impose real costs. Queues impose real costs.

•• Optimize flow rates to minimize WIP and Optimize flow rates to minimize WIP and

to match resource capacity to match resource capacity –– utilization utilization

and efficiency will follow. and efficiency will follow.


4/4/2013

6

The Paradigm ShiftThe Paradigm Shift

��It is counterIt is counter--productive to optimize for productive to optimize for

maximum utilization of the production maximum utilization of the production

resource (developers). resource (developers).

��One cannot reliably predict the production One cannot reliably predict the production

time for individual changes. time for individual changes.


Conflicting Ideas You Might HearConflicting Ideas You Might Hear

•• If we’re going to meet the deadlines in If we’re going to meet the deadlines in

this project, we must keep all of our this project, we must keep all of our

developers busy all the time. developers busy all the time.

•• We can improve our efficiency if we group We can improve our efficiency if we group

all the database changes together and all the database changes together and

give them to our database expert. give them to our database expert.


4/4/2013

7

The Cost of QueuesThe Cost of Queues



•• Minimum average service time comes Minimum average service time comes

from single queue, multiple “all purpose” from single queue, multiple “all purpose”

servers. servers.

•• Provided transaction costs are kept low Provided transaction costs are kept low

enough, reducing packet size improves enough, reducing packet size improves

flow rates. flow rates.

More From

Queueing

Theory


4/4/2013

8

Scaling ChallengesScaling Challenges

Product Owner RoleProduct Owner Role

•• Overwhelming in large productsOverwhelming in large products

•• Must represent Domain SMEs, Architecture, Must represent Domain SMEs, Architecture,

Requirements, Internal and External Requirements, Internal and External

Customers. Customers.

•• Easy to under staffEasy to under staff

•• Intrinsic schedule conflictsIntrinsic schedule conflicts

•• Very critical to process stabilityVery critical to process stability


Waterfall vs. XPWaterfall vs. XP

Change in customer focusChange in customer focus

Under Water FallUnder Water FallUnder Water FallUnder Water FallUnder Water FallUnder Water FallUnder Water FallUnder Water Fall

•• Architecture centralized, Architecture centralized,

set early.set early.

•• Developers relatively Developers relatively

isolated from customer.isolated from customer.

•• Late stage work driven by Late stage work driven by

internal customers.internal customers.

•• Loss of customer focus. Loss of customer focus.

Under XPUnder XPUnder XPUnder XPUnder XPUnder XPUnder XPUnder XP

•• Architecture left to team, Architecture left to team,

set late.set late.

•• Developers sit with Developers sit with

customer.customer.

•• Work and prioritization Work and prioritization

driven by customer.driven by customer.

•• Team alone must protect Team alone must protect

internal standards. internal standards.

Especially with scaled scrum, maintaining a balanced investment in

internally driven requirements (e.g., architectural cohesiveness and

maintainability, for example) may prove difficult. Copyright © 2013, Robert Ward

4/4/2013

9


Managing GrowthManaging Growth

•• Invest in scrum training. Include top Invest in scrum training. Include top

management.management.

•• As much as practical “grow and split”As much as practical “grow and split”

•• The “product team” (architects, product The “product team” (architects, product

owners, requirements analysts) needs owners, requirements analysts) needs

traditional resource management.traditional resource management.



Maintaining Internal IntegrityMaintaining Internal Integrity

•• Architects and SMEs (and even Architects and SMEs (and even

engineering managers) participate in engineering managers) participate in

planning planning –– extend product owner. extend product owner.

•• PM must practice “portfolio balancing”PM must practice “portfolio balancing”•• Technical debtTechnical debt

•• Features for product distinctionFeatures for product distinction

•• Marketplace parity.Marketplace parity.

•• Technical trail blazingTechnical trail blazing

•• Development technologyDevelopment technologyCopyright © 2013, Robert Ward

4/4/2013

10


Configuration ManagementConfiguration Management

•• Branching to insulate teams from Branching to insulate teams from

integration issues is a disaster.integration issues is a disaster.

•• Everyone tries to merge at the end of the Everyone tries to merge at the end of the

sprint.sprint.

•• Nothing works, nothing is ready to ship.Nothing works, nothing is ready to ship.

Absorb all costs

(especially integration and functional test costs)

at the earliest opportunity.



Maintaining Process IntegrityMaintaining Process Integrity

•• Reserve some resource for a “rapid Reserve some resource for a “rapid

response” response” KanbanKanban team.team.

•• Especially with certain customers, as projects Especially with certain customers, as projects

grow in size, demands for “urgent” fixes and grow in size, demands for “urgent” fixes and

other immediate results also grow. These other immediate results also grow. These

demands will become “wolves” savaging your demands will become “wolves” savaging your

scrum teams if you don’t provide some scrum teams if you don’t provide some

mechanism for addressing them. mechanism for addressing them.


4/4/2013

11


Enabling Continuous IntegrationEnabling Continuous Integration

•• Integration APIs must evolve gracefully. Integration APIs must evolve gracefully.

•• Insist on backward compatibility.Insist on backward compatibility.

•• “Server” is always responsible for “Server” is always responsible for

maintaining gateway and endpoint. maintaining gateway and endpoint.

•• Automated E2E functional tests must be Automated E2E functional tests must be

an integral part of continuous integration.an integral part of continuous integration.

•• Cultivate an understanding that merging Cultivate an understanding that merging

is is notnot wasted or unproductive time. wasted or unproductive time.

•• Resist pipelined “integration sprints.” Resist pipelined “integration sprints.” Copyright © 2013, Robert Ward


Architectural AlignmentArchitectural Alignment

•• Code reviews are unreliable. The team that Code reviews are unreliable. The team that

misses the architectural principal when misses the architectural principal when

drafting will also miss it when reviewing. drafting will also miss it when reviewing.

•• Consider creating defined audit points and Consider creating defined audit points and

have a small team conduct continuous code have a small team conduct continuous code

audits for those critical audits for those critical architectural and architectural and

testing compliance testing compliance points. points.


4/4/2013

12

SummarySummary

•• Scrum can significantly improve developer Scrum can significantly improve developer

job satisfaction, but that is a secondary job satisfaction, but that is a secondary

result. Business objectives are driving.result. Business objectives are driving.

•• Scrum works best when management Scrum works best when management

understands the connections to SPC, understands the connections to SPC,

Lean, HPT, Queuing theory, etc. Lean, HPT, Queuing theory, etc.

•• Scrum is not just a “tweak” on traditional Scrum is not just a “tweak” on traditional

management, it is a major paradigm shift.management, it is a major paradigm shift.


ResourcesResources

Deming, William Edwards.Deming, William Edwards. Out of the CrisisOut of the Crisis. MIT press, 2000.. MIT press, 2000.

Buchholz, Steve, Thomas Roth, and Buchholz, Steve, Thomas Roth, and KärenKären M. Hess.M. Hess. Creating the high performance Creating the high performance

teamteam. John Wiley & Sons Inc, 1987.. John Wiley & Sons Inc, 1987.

ReinertsenReinertsen, D. G. , D. G. The Principles of Product Development FlowThe Principles of Product Development Flow––Second Generation Lean Second Generation Lean

Product Development. Product Development. CeleritasCeleritas PublishingPublishing, , 2009.2009.

Carson, Shelley.Carson, Shelley. Your creative brain: Seven steps to maximize imagination, productivity, Your creative brain: Seven steps to maximize imagination, productivity,

and innovation in your lifeand innovation in your life. . JosseyJossey--Bass, 2010.Bass, 2010.

GladwellGladwell, Malcolm., Malcolm. Outliers: The story of successOutliers: The story of success. . ePenguinePenguin, 2008., 2008.

Martin, Robert C.Martin, Robert C. Clean code: a handbook of agile software craftsmanshipClean code: a handbook of agile software craftsmanship. Prentice . Prentice

Hall, 2008.Hall, 2008.

Cohn, Mike.Cohn, Mike. Succeeding with agile: software development using ScrumSucceeding with agile: software development using Scrum. Addison. Addison--Wesley Wesley

Professional, 2009.Professional, 2009.

LarmanLarman, Craig., Craig. Agile and iterative development: a manager's guideAgile and iterative development: a manager's guide. Addison. Addison--Wesley Wesley

Professional, 2004Professional, 2004..

KnibergKniberg, , HenrikHenrik. ". "KanbanKanban vsvs Scrum."Scrum." Crisp AB. Crisp AB. ViitattuViitattu 1 (2009): 2011.1 (2009): 2011.


Documents

SEA - Challenges In Scaling Scrum First...••Scrum can significantly improve developer job satisfaction, but that is a secondary result. Business objectives are driving. ••Scrum