Lean & Agile Methods & Frameworks: Perspectives on Kanban for IT

Perspectives on Kanban for ITDr. David F. Rico, PMP, ACP, CSM

Twitter: @dr_david_f_ricoWebsite: http://www.davidfrico.com

LinkedIn: http://www.linkedin.com/in/davidfricoFacebook: http://www.facebook.com/profile.php?id=1540017424

Dave’s Agile Articles: http://davidfrico.com/agile-message.doc

Lean & Agile Methods & Frameworks

Author Background DoD contractor with 28+ years of IT experience B.S. Comp. Sci., M.S. Soft. Eng., & D.M. Info. Sys. Large gov’t projects in U.S., Far/Mid-East, & Europe

2

Published six books & numerous journal articlesAdjunct at George Wash, UMBC, UMUC, ArgosyAgile Program Management & Lean DevelopmentSpecializes in metrics, models, & cost engineeringSix Sigma, CMMI, ISO 9001, DoDAF, & DoD 5000Cloud Computing, SOA, Web Services, FOSS, etc.

Today’s Whirlwind Environment

3

OverrunsAttritionEscalationRunawaysCancellation

GlobalCompetition

DemandingCustomers

OrganizationDownsizing

SystemComplexity

TechnologyChange

VagueRequirements

Work LifeImbalance

InefficiencyHigh O&MLower DoQVulnerableN-M Breach

ReducedIT Budgets

81 MonthCycle Times

RedundantData Centers

Lack ofInteroperability

PoorIT Security

OverburdeningLegacy Systems

ObsoleteTechnology & Skills

Pine, B. J. (1993). Mass customization: The new frontier in business competition. Boston, MA: Harvard Business School Press. Pontius, R. W. (2012). Acquisition of IT: Improving efficiency and effectiveness in IT acquisition in the DoD. Second Annual AFEI/NDIA Conference on Agile in DoD, Springfield, VA, USA.

Traditional Projects

4

Big projects result in poor quality and scope changes Productivity declines with long queues/wait times Large projects are unsuccessful or canceled

Jones, C. (1991). Applied software measurement: Assuring productivity and quality. New York, NY: McGraw-Hill.

Size vs. Quality

Def

ect

Den

sity

0.00

3.20

6.40

9.60

12.80

16.00

0 2 6 25 100 400

Lines of Code (Thousands)

Size vs. Productivity

Cod

e P

rodu

ctio

n R

ate

0.00

1.00

2.00

3.00

4.00

5.00

0 2 6 25 100 400


Size vs. Requirements Growth

Per

cent

age

0%

8%

16%

24%

32%

40%

0 2 6 25 100 400


Size vs. SuccessP

erce

ntag

e

0%

12%

24%

36%

48%

60%

0 2 6 25 100 400


Global Project Failures

5Standish Group. (2010). Chaos summary 2010. Boston, MA: Author.Sessions, R. (2009). The IT complexity crisis: Danger and opportunity. Houston, TX: Object Watch.

Challenged and failed projects hover at 67% Big projects fail more often, which is 5% to 10% Of $1.7T spent on IT projects, over $858B were lost

16% 53% 31%

27% 33% 40%

26% 46% 28%

28% 49% 23%

34% 51% 15%

29% 53% 18%

35% 46% 19%

32% 44% 24%

33% 41% 26%

0% 20% 40% 60% 80% 100%

1994

1996

1998

2000

2002

2004

2006

2008

2010

Year

Successful Challenged Failed

$0.0

$0.4

$0.7

$1.1

$1.4

$1.8

2002 2003 2004 2005 2006 2007 2008 2009 2010

Trill

ions

(US

Dolla

rs)

Expenditures Failed Investments

Requirements Defects & Waste

6Sheldon, F. T. et al. (1992). Reliability measurement: From theory to practice. IEEE Software, 9(4), 13-20Johnson, J. (2002). ROI: It's your job. Extreme Programming 2002 Conference, Alghero, Sardinia, Italy.

Requirements defects are #1 reason projects fail Traditional projects specify too many requirements More than 65% of requirements are never used at all

Other 7%

Requirements47%

Design28%

Implementation18%

Defects

Always 7%

Often 13%

Sometimes16%

Rarely19%

Never45%

Waste

What is Agility? A-gil-i-ty (ə-'ji-lə-tē) Property consisting of quickness,

lightness, and ease of movement; To be very nimble The ability to create and respond to change in order to

profit in a turbulent global business environment The ability to quickly reprioritize use of resources when

requirements, technology, and knowledge shift A very fast response to sudden market changes and

emerging threats by intensive customer interaction Use of evolutionary, incremental, and iterative delivery

to converge on an optimal customer solution Maximizing BUSINESS VALUE with right sized, just-

enough, and just-in-time processes and documentationHighsmith, J. A. (2002). Agile software development ecosystems. Boston, MA: Addison-Wesley.

7

What are Agile Methods?

8

People-centric way to create innovative solutions Product-centric alternative to documents/process Market-centric model to maximize business value

Agile Manifesto. (2001). Manifesto for agile software development. Retrieved September 3, 2008, from http://www.agilemanifesto.orgRico, D. F., Sayani, H. H., & Sone, S. (2009). The business value of agile software methods. Ft. Lauderdale, FL: J. Ross Publishing.Rico, D. F. (2012). Agile conceptual model. Retrieved February 6, 2012, from http://davidfrico.com/agile-concept-model-1.pdf

Customer Collaboration

Working Software

Individuals & Interactions

Responding to Change

valuedmore than

valuedmore than

valuedmore than

valuedmore than

Contracts

Documentation

Processes

Project Plans

Frequent comm. Close proximity Regular meetings

Multiple comm. channels Frequent feedback Relationship strength

Leadership Boundaries Empowerment

Competence Structure Manageability/Motivation

Clear objectives Small/feasible scope Acceptance criteria

Timeboxed iterations Valid operational results Regular cadence/intervals

Org. flexibility Mgt. flexibility Process flexibility

System flexibility Technology flexibility Infrastructure flexibility

Contract compliance Contract deliverables Contract change orders

Lifecycle compliance Process Maturity Level Regulatory compliance

Document deliveries Document comments Document compliance

Cost Compliance Scope Compliance Schedule Compliance

NetworkComputer

Operating SystemMiddlewareApplications

APIsGUI

How Agile Works Agile requirements implemented in slices vs. layers User needs with higher business value are done first Reduces cost & risk while increasing business success

9Shore, J. (2011). Evolutionary design illustrated. Norwegian Developers Conference, Oslo, Norway.

Agile Traditional1 2 3 Faster

Early ROI

Lower Costs

Fewer Defects

Manageable Risk

Better Performance

Smaller Attack Surface

Late

No Value

Cost Overruns

Very Poor Quality

Uncontrollable Risk

Slowest Performance

More Security Incidents Seven Wastes1.Rework2.Motion3.Waiting4.Inventory5 .Transportation6.Overprocessing7 .Overproduction

MINIMIZES MAXIMIZES

JIT, Just-enough architecture Early, in-process system V&V Fast continuous improvement Scalable to systems of systems Maximizes successful outcomes

Myth of perfect architecture Late big-bang integration tests Year long improvement cycles Breaks down on large projects Undermines business success

Thousands of TestsContinuously Executed

No More Late BigBang Integration

Agile Development Model User needs designed & developed one-at-a-time Changes automatically detected, built, and tested System fully tested and deployed as changes occur

10Humble, J., & Farley, D. (2011). Continuous delivery. Boston, MA: Pearson Education.Duvall, P., Matyas, S., & Glover, A. (2006). Continuous integration. Boston, MA: Addison-Wesley.

BuildIntegration

Server

VersionControlServer

BuildScripts

UsesWatches

BuildStatus

ProvidesDeveloper A

Developer B

Developer C

CommitsChanges

CommitsChanges

CommitsChanges

Builds

Database

Analysis

Testing

Reporting

Documentation

Deployment

Early, Automated, Fast,Efficient, & Repeatable

Constant ReadinessState & CM Control

Lean, Waste Free, Low WIP,No Deadlocked Test Queues

Rapidly & SuccessfullyDev. Complex Systems

How Do Lean & Agile Intersect?

11

Agile is naturally lean and based on small batches Agile directly supports six principles of lean thinking Agile may be converted to a continuous flow system

Womack, J. P., & Jones, D. T. (1996). Lean thinking: Banish waste and create wealth in your corporation. New York, NY: Free Press.Reinertsen, D. G. (2009). The principles of product development flow: Second generation lean product development. New York, NY: Celeritas.Reagan, R. B., & Rico, D. F. (2010). Lean and agile acquisition and systems engineering: A paradigm whose time has come. DoD AT&L Magazine, 39(6).

Economic View

Decentralization

Fast Feedback

Control Cadence& Small Batches

Manage Queues/Exploit Variability

WIP Constraints& Kanban

Flow PrinciplesAgile Values

CustomerCollaboration

EmpoweredTeams

IterativeDelivery

Respondingto Change

Lean Pillars

Respectfor People

ContinuousImprovement

Customer Value

Relationships

Customer Pull

Continuous Flow

Perfection

Value Stream

Lean Principles Customer relationships, satisfaction, trust, and loyalty Team authority, empowerment, and resources Team identification, cohesion, and communication

Lean & Agile Practices

Product vision, mission, needs, and capabilities Product scope, constraints, and business value Product objectives, specifications, and performance As is policies, processes, procedures, and instructions To be business processes, flowcharts, and swim lanes Initial workflow analysis, metrication, and optimization Batch size, work in process, and artifact size constraints Cadence, queue size, buffers, slack, and bottlenecks Workflow, test, integration, and deployment automation Roadmaps, releases, iterations, and product priorities Epics, themes, feature sets, features, and user stories Product demonstrations, feedback, and new backlogs Refactor, test driven design, and continuous integration Standups, retrospectives, and process improvements Organization, project, and process adaptability/flexibility

What is Lean Development? Lean (lēn): Thin, slim, slender, narrow, adequate,

or just-enough; Without waste A customer-driven product development process that

delivers the maximum amount of business value An economical way of planning and managing the

development of complex new products and services A product development process that is free of excess

waste, capacity, and non-value adding activities Just-enough, just-in-time, and right-sized product

development processes, documentation, and tools A product development approach that is adaptable to

change in customer needs and market conditions

12

Womack, J. P., & Jones, D. T. (1996). Lean thinking: Banish waste and create wealth in your corporation. New York, NY: Free Press.Liker, J. K. (2004). The toyota way: 14 management principles from the world’s greatest manufacturer. New York, NY: McGraw Hill.Larman, C., & Vodde, B. (2008). Scaling lean and agile development: Thinking and organizational tools for large-scale scrum. Boston, MA: Addison-Wesley.

Lean Thinking

13

Term coined by John Krafcik of MIT in 1988 Taiichi Ohno of Toyota is credited with its ideas Toyota Production System was adapted from Ford

Womack, J. P., & Jones, D. T. (1996). Lean thinking: Banish waste and create wealth in your corporation. New York, NY: Free Press.Liker, J. K. (2004). The toyota way: 14 management principles from the world’s greatest manufacturer. New York, NY: McGraw Hill.Larman, C., & Vodde, B. (2008). Scaling lean and agile development: Thinking and organizational tools for large-scale scrum. Boston, MA: Addison-Wesley.

Lean Six Sigma

14

Created in late 1990s by Allied Signal and Maytag Combination of Six Sigma and Lean Thinking Focuses on eliminating waste vs. variation

George, M. L. (2002). Lean six sigma: Combining six sigma quality with lean speed. New York, NY: McGraw-Hill.

Lean Product Development

15

Lean product development emerged in the 1980s Adaptation of Toyota Production System (TPS) “Toyota [New] Product Development System”

Clark, K. B., & Fujimoto, T. (1991). Product development performance: Strategy, organization, and management in the world auto industry. Boston, MA: Harvard Business School Press.

Lean Development

Frequent set-up changes

Lean Manufacturing

Short manufacturing throughput time

Reduced work-in-process inventory between manufacturing steps

Frequent transfer of small batches of parts between manufacturing steps

Reduced inventory requires slack resources and more information flow between steps

Adaptability to changes in volume, product mix, and product design

Broad task assignments for production workers gives higher productivity

Focus on quick problem solving and continuous process improvement

Simultaneous improvement in quality, delivery time, and manufacturing productivity

Frequent product changes

Short development time

Reduced information inventory between development steps

Frequent transfer of preliminary informationbetween development steps

Reduced development time requires slack resources and information flow between stages

Adaptability to changes in product design, schedule, and cost targets

Broad task assignments for engineers (developers) gives higher productivity

Focus on frequent incremental innovation and continuous product and process improvement

Simultaneous improvement in quality, development time, and development productivity

Lean Systems Engineering

16

Origin in MIT Lean Aerospace Initiative in 1992 Lean Systems Engineering WG formed in 2006 Lean Enablers for Systems Engineering in 2009

INCOSE. (2009). Lean enablers for systems engineering. Retrieved October 20, 2009, from http://www.incose.org/practice/techactivities/wg/leansewg.

Value

Map the Value

Stream

Flow

Pull

Perfection

Respect for People

Customer involvement

Streamline developmentDeconflict suppliersEstablish metrics

Communicate effectivelyAchieve smooth flowEnsure program visibilityUse lead

Pull tasks as-needed

Appoint a chiefEliminate wastePerform continuous improvement

Create a learning environmentTreat people as valuable assets

Requirements engineeringBusiness value analysis

Program planningMap value streamsFrontload program

Program monitoringClarify requirementsFrontload architectureCoordinate activities

Tailor program

Continuous improvementStrive for excellencePerform lessons learnedDevelop communication plan

Human resourcesRespect all peopleStrive for technical excellence

What is Kanban? Kan-ban ('kæn-bæn): Signboard, billboard, signal

cards; Lean, just-in-time system of production A lean and just-in-time manufacturing process for

regulating the flow of production based on demand A pull-system philosophy of customized production vs.

a push system of mass-market manufacturing A set of principles for creating a lean, efficient, and

waste-free product flow by limiting work-in-process Use of simple organizational policy changes resulting

in order-of-magnitude performance improvements Framework for optimizing workflow that maximizes

efficiency, product quality, and customer satisfaction

17

Kniberg, H., & Skarin, M. (2010). Kanban and scrum: Making the most of both. Toronto, ON: C4 Media, Inc.Ladas, C. (2008). Scrumban: Essays on kanban systems for lean software development. Seattle, WA: Modus Cooperandi.Anderson, D. J. (2010). Kanban: Successful evolutionary change for your technology business. Sequim, WA: Blue Hole Press.

Kanban for IT Projects

18

Adapted to IT by Dave Anderson in 2006 Activities, buffers, queues, WIP limits, tasks, etc. Lean, JIT pull/demand system leading to high quality

Anderson, D. J. (2010). Kanban: Successful evolutionary change for your technology business. Sequim, WA: Blue Hole Press.

Kanban Goals

19

Kanban initially seeks to change as little as possible Change without resistance is the first Kanban goal Focus on improving quality, lead time and morale

Goal 2

Goal 3

Goal 4

Goal 5

Goal 6

Goal 7

Goal 8

Deliver high product quality (to build stakeholder trust)

Reduce long lead times (and stabilize them)

Achieve sustainable pace (work-life balance)

Provide process slack (for process improvement)

Simplify workload prioritization (of customer needs)

Provide transparency (into design and operations)

Strive for process maturity (to improve performance)

Goal 1 Optimize existing processes (rather than change them)


Kanban Recipe for Success

20

Based on principles for product development flow Uses operations and mathematical queue theory Pragmatic operating principles for development

Focus on Quality Reduce WIP Deliver Often Balance Demand Prioritize Attack Variability

Walkthroughs

Inspections

Technical reviews

Peer reviews

Pair programming

Test driven design

Continuous integration

Design patterns

Refactoring

Design simplicity

Usability engineering

Formal methods

Process flowcharts

Workflow analysis

Kanban boards

Limit work tasks

Limit queues

Limit buffers

Limit backlogs

Simple prioritization

Adequate resources

Process automation

Policy statements

Simplify process

Short releases

Short increments

Short iterations

Small releases

Frequent releases

Small batch sizes

Customer collaboration

Developer collaboration

Ample communication

Frequent builds

Deploy often

Automatic updates

Regulate inputs

Identify bottlenecks

Create slack

Limit work-in-process

Create pull system

Focus on precision

Focus on quality

Take pride in work

Improve morale

Learn new skills

Obtain training

Continuously improve

Prioritize inputs

Business focus-

Business value focus

Influence prioritization

Stabilize process

Build stakeholder trust

Perform risk analysis

Analyze demand

Evaluate size

Evaluate complexity

Market forecasting

Technology analysis

Work item size

Work item type mix

Service class mix

Irregular flow

Rework

Ambiguous reqmnts.

Expedited requests

Environment avail.

Market fluctuations

Coordination

Technological change

Skill/experience mix


Value Stream Mapping

21

Start by flow-charting the as-is product workflow Add buffers and queues one feels are necessary Add WIP limits to buffers, queues, and activity


22

Traditional vs. Agile Cumulative Flow

Wor

k (S

tory

, Poi

nt, T

ask)

or E

ffor

t (W

eek,

Day

, Hou

r)

Time Unit (Roadmap, Release, Iteration, Month, Week, Day, Hour, etc.)

Wor

k (S

tory

, Poi

nt, T

ask)

or E

ffor

t (W

eek,

Day

, Hou

r)

Time Unit (Roadmap, Release, Iteration, Month, Week, Day, Hour, etc.)

Traditional Cumulative Flow Lean & Agile Cumulative Flow

Anderson, D. J. (2004). Agile management for software engineering. Upper Saddle River, NJ: Pearson Education.Anderson, D. J. (2010). Kanban: Successful evolutionary change for your technology business. Sequim, WA: Blue Hole Press.

High work-in-process leads to longest lead times Low work-in-process greatly reduces lead times Results in better customer trust and satisfaction

Work-in-Process

Scaled Lean & Agile Framework

Beck, K., & Fowler, M. (2001). Planning extreme programming. Upper Saddle River, NJ: Addison-Wesley.Highsmith, J. A. (2010). Agile project management: Creating innovative products. Boston, MA: Pearson Education.Larman, C., & Vodde, B. (2010). Practices for scaling lean and agile development. Boston, MA: Addison-Wesley.Leffingwell, D. (2011). Agile software requirements: Lean requirements practices for teams, programs, and the enterprise. Boston, MA: Pearson Education.

Begins with a high-level product vision/architecture Continues with needs development/release planning Includes agile delivery teams to realize business value

23

Lean & Agile Hybrid Methods

24Shore, J., & Warden, S. (2008). The art of agile development. Sabastopol, CA: O'Reilly Media.Cockburn, A. (2007). Agile software development: The cooperative game. Boston, MA: Addison-Wesley.Shalloway, A., Beaver, G., & Trott, J. R. (2010). Lean-agile software development. Boston, MA: Addison-Wesley.

Scrum is a basic agile project management model Scrum does not specify other critical functions Scrum hybrids include necessary practices

EnterprisePortfolio analysisEarned value mgt.Enterprise arch.Governance

ScrumSprint PlanningDaily ScrumSprint ReviewRetrospective

XPRelease planningPair programmingContinuous integ.Open workspaces

Open SourceEnvironmentsComponentsFrameworksOperating systems

OtherUser exp.Security eng.Reliability eng.Safety analysis

Lean & Agile Recap Agile methods DON’T mean deliver it now & fix it later Lightweight, yet disciplined approach to development Reduced cost, risk, & waste while improving quality

25Rico, D. F. (2012). What’s really happening in agile methods: Its principles revisited? Retrieved June 6, 2012, from http://davidfrico.com/agile-principles.pdfRico, D. F. (2012). The promises and pitfalls of agile methods. Retrieved February 6, 2013 from, http://davidfrico.com/agile-pros-cons.pdfRico, D. F. (2012). How do lean & agile intersect? Retrieved February 6, 2013, from http://davidfrico.com/agile-concept-model-3.pdf

What How ResultFlexibility Use lightweight, yet disciplined processes and artifacts Low work-in-process

Customer Involve customers early and often throughout development Early feedback

Prioritize Identify highest-priority, value-adding business needs Focus resources

Descope Descope complex programs by an order of magnitude Simplify problem

Decompose Divide the remaining scope into smaller batches Manageable pieces

Iterate Implement pieces one at a time over long periods of time Diffuse risk

Leanness Architect and design the system one iteration at a time JIT waste-free design

Swarm Implement each component in small cross-functional teams Knowledge transfer

Collaborate Use frequent informal communications as often as possible Efficient data transfer

Test Early Incrementally test each component as it is developed Early verification

Test Often Perform system-level regression testing every few minutes Early validation

Adapt Frequently identify optimal process and product solutions Improve performance

Conclusion

26

Agility is the evolution of management thought Confluence of traditional and non-traditional ideas Improve performance by over an order of magnitude

“The world of traditional methods belongs to yesterday”“Don’t waste your time using traditional methods on 21st century projects”

Agile methods are …

Systems development approachesNew product development approachesExpertly designed to be fast and efficientIntentionally lean and free of waste (muda) Systematic highly-disciplined approachesCapable of producing high quality systemsRight-sized, just-enough, and just-in-time tools

Scalable to large, complex mission-critical systems Designed to maximize business value for customers

Wysocki, R.F. (2010). Adaptive project framework: Managing complexity in the face of uncertainty. Boston, MA: Pearson Education.

Books on ROI of SW Methods Guides to software methods for business leaders Communicates business value of software methods Rosetta stones to unlocking ROI of software methods

http://davidfrico.com/agile-book.htm (Description) http://davidfrico.com/roi-book.htm (Description)

27