Lean - Agile Project Management - ASPE â€“ SDLC

© 2005-2008 EvolutionTen Corp.

Lean Six Sigma and Agile Project Management

How Lean Process CanSupercharge

Agile Project Management

Lean - Agile Project Management

Presented ByRichard Perrin PMP CSM SSBB QFDGB

ASPE Course Director: Lean-Agile Project Management

The fundamental disconnect >> adoption of the waterfall approach for managing software development was based on a misinterpretation of information:

Requirements Analysis Design Code Test Deploy

* Proceedings, IEEE WESCON, August 1970, pages 1-9.Copyright © 1970 Institute of Electrical and Electronics Engineers

“Waterfall” Software Approach

** “I believe in this concept, but the implementation described above is risky and invites failure... The testing phase which occurs at the end of the development cycle is the first event for which timing, storage, input/output transfers, etc., are experienced as distinguished from analyzed. These phenomena are not precisely analyzable.*

Winston Royce described the model above in a 1970 presentation to the IEEE. He thought the process would work only if iterative feedback loops were built into the process**



Frederick Brooks (author of the “Mythical Man Month”) described Design-Development-Test ratios that would effectively produce well designed, robust and well tested code:

Design Code Test


The Testing Problem

50%

Or – a ratio of 3 to 1 to 2: Design-Code-TestHow many businesses actually develop and test to the above ratios?

16 1/3% 33 2/3%



The waterfall model assumes that new problems can be solved in one pass – ‘get it right the first time’... The issue: you will never get a complete set of requirements from the customerAttempting to solve a new problem is essentially an R & D activity! R&D is by definition a high risk effort fraught with numerous unknowns that require iteration and feedbackDefinition: “A project is a temporary endeavor undertaken to create a unique product, service, or result.”* “Unique”according to Webster means “being without a like or equal”.

* A Guide to the Project Management Body of Knowledge, 3rd Edition, 2004

The Project Problem



Why is building software ‘hard’? (Or is it?)◦ Software developers function in a permanent state of emergency◦ Businesses reward developers who deliver software that is cheap,

buggy, and first◦ People want fancy new gadgets now:

They don't want inconvenienceThey don't want to learn new ways of interacting with their computers They don't want delays in delivery They don't want to pay extra for quality

◦ Without real changes in user behavior, software suppliers are unlikely to change*

◦ Software engineering is an applied science – do your users think so?

* The Problem with Programming, Nov. 2006 interview w/ Bjarne Stroustrup, the inventor of the C++ programming language, by Jason Pontin, Technology Review, MIT Press

The Other Project Problem: Building Software Is Difficult



The second disconnect >> most business users think building software is like:◦ Building a house ◦ Ordering fast food


Why?◦ PMI was founded on construction industry principles

“Waterfall” Software Approach



If you are building a house and you need an estimate for roofing, drywall, foundation, electrical, plumbing, etc. where do you go for the estimate? ◦ RS Means: CostWorks. CD-ROM that

prices 23,500 building components by geography and skill set

If you are estimating a software project:◦ What guide exists to estimate software

engineering cost?◦ What are the ‘standard jobs’ in

creating software?

Building a House vs. Building Software



The lean Toyota TPS process borrows heavily from the empirical process defined by Dr. Walter Shewhart of Western Electric in the late 1930’s.The Plan-Do-Check-Act cycle or as it is now known, the Shewhart-Deming cycle, exemplifies the incremental improvement process for all manufacturing and service businessesSCRUM is fundamentally a P-D-C-A cycle

Defining an Empirical Process

Plan

Do

Check

Act

Without a theory of reality, there is no plan –everything starts with a theory. Your plan becomes the vehicle for testing your theory

Based on the theory we do the work as defined in the plan – execute the plan accurately and with precision. We then check the

result to see if what we thought would occur actually happened. What worked – what didn’t?

We hold the gains from the elements that worked. We make adjustments for the elements that didn’t work and roll it, plus new elements, into the next plan



Product Backlog

DailyStand-up:15 mins

Sprint Cycle:15 or 30 days

PotentiallyShippableProduct

Increment

Prioritized by Product Owner

1. What you did yesterday2. What you will do today3. What are your obstacles?

Team:Based on negotiation withProduct Owner, selects highest priority work from backlog

SCRUM Basics



Five fundamental principles:Specify value by productIdentify the value streamMake value flowCustomer ‘pulls’ value from the producerPursue perfection

How does an Agile process (e.g. SCRUM) play into Lean (and vice-versa)?



Lean Concepts

Specify Value – defined by the ultimate customer and expressed in terms of needs, price, time and quality Identify the value stream – activities

required to bring products to customers

Flow – encourages everyone to abandon the batch-and-queue mode of thinking: keep activities flowing without delays, approvals, etc.

Pull – wait for the customer to ask for the product instead of building a large inventory and pushing the product to the customer

Four Rules of Lean Thinking



Waste (muda) in process consists of the following:1. Excessive Inventory2. Extra Motion3. Repetitive Processing4. Waiting/Delay5. Excessive Transport6. Overproduction7. Repair/rejects/defects

Lean process addresses the first five elements in the listSix Sigma addresses the final two elements



The Seven Wastes

The work may be:◦ Coded◦ Unit and Component tested◦ DocumentedNo guarantee it will work until final Integration/System testingIf the project is cancelled (for other reasons), what happens to this work investment?




Inventory: Partially Done Work

Does all the paperwork add value?◦ “Quality documents” (Paperwork hides defects)◦ Required sign-offs◦ Thick, heavy, binders loaded with a prescriptive

project management methodology that must be followed

Administrative meetings Top heavy compliance processes (Is the process more work than the work?)Time and money spent on risk planning for low probability events? (i.e. avoidance of real risk assessment)

Extra Processing



Does multi-tasking work?◦ According to the TOC (Theory of Constraints) multi-

tasking is the worst way to meet multiple business needs

◦ Percent of time spent on value-added activity drops rapidly when a developer is working on more than two tasks

Concurrently Assigned Tasks

Productivity**

1 70%

2 80%

3 60%

4 47%

5 39%

**Managing New Product and Process Development: Text and Cases, Clark and Wheelwright,1993, Free Press



Transportation – Task Switching

Delay infuriates the customerExample: IVR Process

Forcing the customer to go through an IVR (Interactive Voice Response) unit instead of directing the customer to a Customer Service Representative (CSR)The unit responds with “You have pushed an incorrect key” when repeatedly hitting ‘0’ to connect with the CSRAfter entering information, you are asked for it again when the CSR finally answersThroughout this process you are told that you are a valued customer (are you?)

Frustrated customers go to www.gethuman.com

Delay: Waiting, Waiting…



Where does the team have to go for answers to:◦ Development questions? (The

other end of the building?)◦ Business questions? (Three

hours tracking down a resource that has critical information)

◦ If the developer is interrupted while working, how long does it take to get back into ‘flow’?

How does information flow from architects to designers to developers? ◦ Do the hand-offs work well?◦ Is the documentation

complete?

13'-6"

8'-0

"

13'-6"

8'-0

"

13'-6"

19'-0

"

13'-6"

15'-6

"

13'-6"

8'-0

"

13'-6"

19'-6

"

9'-8

"

18'-6"

12'-1

0"

4'-0"

22'-6

"

22'-6"

18'-6"

12'-6

"36

'-0"

????5'-3"

2'-6"

2'-6

"

2'-6"

2'-6

"

2'-6

"

5'-0

"

3'-6

"

2'-6

"5'

-0"

2'-6

"



Unnecessary Motion

Giving the customer something they did not request (They will wonder what you are doing with the rest of their budget…)Features added by the development team because they think the features are ‘cool’. (Does the customer think so?)

Gold Plating: Over-Production



The later defects are found in the development/testing process, the more expensive they become to fixDefects that are missed in system test and delivered to the customer yield:◦ The highest cost to repair◦ Loss of face and credibility to the

customer◦ Potential loss of businessRemember Ashton-Tate?

Defects: The Ultimate Dissatisfier



What elements of the software process are value added from the customer’s perspective?


? ? ? ? ?

Winston Royce stated only _____ of the above elements are considered “value added” by the customer – what are they?*


Customer Value



The Toyota Production System (TPS) exemplifies lean processing and manufacturingA number of the concepts and practices of Agile directly map to the TPSThe key 14 points in the TPS (with selected sub-topics):

1. Base your management decisions on a long-term philosophy, even at the expense of short-term financial goals

Generate value for the customer, the society and the economy2. Create continuous process flow to bring problems to the surface

Re-design work processes to achieve high value-added, continuous flow. Strive to cut back to zero the mount of time that any work project is sitting idle or waiting for someone to work on it.

3. Use "pull" systems to avoid overproduction.Provide your downline customers in the production process with what they want, when they want it, and in the amount they want.Be responsive to the day by day shifts in customer demand rather than relying on computer schedules and systems to track wasteful inventory.

Lean Exemplified: Toyota Production System



4. Level out the workload.◦ Eliminating waste is just one third of the equation for making lean successful.◦ Work to level out the workload of all manufacturing and service processes as an alternative to the

stop/start approach of working on projects in batches that is typical of most companies.5. Build a culture of stopping to fix problems, to get quality right the

first time.◦ Quality for the customer drives your value proposition. ◦ Build into your processes the capability of detecting problems and stopping itself.◦ Build into your culture the philosophy of stopping or slowing down to get quality right the first time

to enhance productivity in the long run.6. Standardized tasks are the foundation for continuous improvement

and employee empowerment.◦ Use stable, repeatable methods everywhere to maintain predictability, regular timing, and regular

output of your processes.◦ Capture the cumulative learning about a process up to a point in time by standardizing today's

best practices. Allow creativity and individual expression to improve upon the standard; then incorporate into the new standards so that when a person moves on you can hand off the learning to the next person.

7. Use visual control so no problems are hidden.◦ Use simple visual indicators to help people determine immediately whether they are in a standard

condition or deviating from it.◦ Design simple visual systems at the place where work is done, to support flow and pull.◦ Reduce your reports to one piece of paper whenever possible, even for your most important

financial decisions.

TPS: Points 4-7



8. Use only reliable, thoroughly tested technology that serves your people and processes.

◦ Use technology to support people, not replace people. Often it is best to work out a process manually before adding technology to support the process.

◦ Reject or modify technologies that conflict with your culture or that might disrupt stability, reliability, and predictability.

◦ Nevertheless, encourage your people to consider new technologies when looking into new approaches to work.

9. Grow leaders who thoroughly understand the work, live the philosophy, and teach it to others.

◦ Do not view the leader's job is simply accomplishing tasks and having good people skills. Leaders must be role models of the company's philosophy and way of doing business.

◦ A good leader must understand the daily work in great detail so he or she can be the best teacher of your company's philosophy.

10. Develop exceptional people and teams to follow your company's philosophy.

◦ Train exceptional individuals and teams to work within the corporate philosophy to achieve exceptional results. Work very hard to reinforce the culture continually

◦ Use cross functional teams to improve quality and productivity and enhance flow by solving difficult technical problems.

11. Respect your extended network of partners and suppliers by challenging them and helping them improve.

◦ Challenge your outside business partners to grow and develop. It shows you value them. Set challenging targets and assist your partners in achieving them.

TPS: Points 8-11



12. Go and see for yourself to thoroughly understand the situation.◦ Solve problems and improve processes by going to the source and personally observing

and verifying data rather than theorizing on how the basis of what other people or the computer screen tell you.

13. Make decisions slowly by consensus, thoroughly considering all options; implement decisions rapidly.◦ Do not take a single direction and go down that one path until you have thoroughly

considered alternatives

14. Become a learning organization through relentless reflection andcontinuous improvement.◦ Once you have established a stable process, use continuous improvement tools to

determine the root cause of inefficiencies and apply effective countermeasures.◦ Design processes that require almost no inventory. This will make waste of time and

resources visible for all to see.

TPS: Points 12-14



Developed by Toyota engineer Shigeo ShingoOne step beyond SPC (Statistical Process Control) – why was it developed?◦ Shingo realized that SPC measured the process after

the fact - e.g. it’s great to have safety sprinklers in the ceiling to help put out a fire. However - isn’t it better to prevent the fire in the first place?

◦ ZQC is 100% source inspection: any issue/out-of-control condition is addressed and fixed immediately.

What is ZQC: Zero Quality Control?



Three inspection methods:◦ Inspections that discover defects: judgment

inspectionsCategorizes finished products as defective or acceptable after processing is completedIt does not lower the initial defect rateHighest cost due to rework or scrap

◦ Inspections that reduce defects: informative inspections

Uses Statistical Quality Control (SQC) by feeding back data about defects to reduce future defectsToo slow to be fully effective

Zero Quality Control & Inspection Systems

**Zero Quality Control: Source Inspection and the Poka-Yoke System, Shigeo Shingo,1986, Productivity Press




Three inspection methods - continued:◦ Inspections that eliminate defects: source inspections

Preventing an error from becoming a defectImmediate feedback enables immediate action to be taken on the error**Software implementations: rolling wave integration test, TDD

**Zero Quality Control: Source Inspection and the Poka-Yoke System, Shigeo Shingo,1986, Productivity Press

Iterations

Delivered Product Increments

User feedback (UF) UF UF UF



Zero Quality Control & Inspection Systems

Plan

Do

Check

Act

Plan

Do

Check

Act

Plan

Do

Check

Act

Plan

Do

Check

Act

Traditional project management defines quality as a function of the Triple Constraint concept:

Scope

Time Budget

Q

Does this address the needs of the customer?Does this triple constraint address ‘value’ in the customer’s mind?



What Is Quality?

Quality as defined by:◦ PMI >>PMBOK 2004: “The degree to which a set of inherent

characteristics fulfills requirements”◦ ISO 8402: “The totality of characteristics of an entity that bear

on its ability to satisfy stated and implied needs”◦ Peter Drucker (Innovation and Entrepreneurship):

“Quality in a product or service is not what the supplier puts in. It is what the customer gets out and is willing to pay for. A product is not quality because it is hard to make and costs a lot of money, as manufacturers typically believe. This is incompetence. Customers pay only for what is of use to them and gives them value. Nothing else constitutes quality.”

Which definition sounds the best to you?Which definition would satisfy your customer the best?

1-29© 2005-2008 EvolutionTen Corp.


Some Definitions of Quality

Lean – agile process focuses on customer satisfaction and customer defined valueWaterfall focuses on the satisfaction of the Triple-Constraint (Scope-Budget-Timeline) - does this determine ultimate customer satisfaction?◦ “It was not the fair market value of inventories, receivables, or fixed assets

that produce See’s Candies premium rates of return, rather it was a combination of intangible assets, particularly a pervasive favorable reputation with consumers based on countless pleasant experiences they had with both product and personnel. Such a reputation creates a franchise that allows the value of the product to the purchaser, rather than the production cost, to be the major determinant of selling price.“

Berkshire Hathaway annual report of 1983

1-30

Lean vs. Waterfall





Agile Principles

Supporting Agile Principles to the Agile Manifesto:

• Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage

• Customer collaboration over contract negotiation• Business people and developers must work together daily throughout

the project• Build projects around motivated individuals. Give them the environment

and support they need, and trust them to get the job done • The most efficient and effective method of conveying information to and

within a development team is face-to-face conversation • At regular intervals, the team reflects on how to become more effective,

then tunes and adjusts its behavior accordingly *

* http://www.agilealliance.org

Agile Principles –Cont’d



The TPS exemplifies the Lean process – how do lean and agile support the same principles?

Both exemplify an empirical process (Plan-Do-Check-Act)Both build quality into the product or service instead of inspecting it into the product at the end of the development cycle.Both focus on continuous improvement.Face-to-face communications are more important than e-mails or what the user sees on a computer screen.Engage highly motivated individuals to get the job done i.e. get ‘the right people on the bus’.Reflection is a key element in helping the team to continuously evaluate and improve its effectiveness.Live and breath the philosophy of improvement and be an example to your employees, your customers and your vendors

Agile-Lean Synergies



Waterfall-Agile Comparison



Q & AResources◦ “Agile & Iterative Development”, Craig Larman◦ “Agile Software Development”, Alistair Cockburn◦ “Agile Software Development with Scrum”, Ken Schwaber and

Mike Beedle◦ “Agile Project Management with Scrum”, Ken Schwaber◦ “Agile Estimating and Planning”. Mike Cohn◦ “Agile Project Management”, Jim Highsmith◦ “Software Endgames”, Robert Galen◦ “Lean Software Development”, Tom and Mary Poppendeick◦ “The Toyota Way” , Jeffrey K Liker◦ “Real World Project Management” Richard Perrin

Thank You!



Q&A Session for Lean – Agile Software Project Management Web seminar on February 25, 2008 – Richard Perrin 1. How does the iterative process of Agile affect the project scope and ultimate project completion date? It appears this methodology opens the door for scope creep & a more expensive project more so than with the Waterfall approach. First if the project elements are very well understood and there is very little impact from uncertain, unknown or high-risk elements a waterfall approach may work well for you (we don’t throw the baby out with the bathwater). However if there is a significant impact from unknown elements or simply ‘you don’t know what you don’t know’, the idea that you can accurately scope and define the ultimate project completion date 9 months or year down the road is quite frankly a fantasy… The notion of ‘scope creep’ doesn’t compute in an Agile environment. Also, see response to #13. 2. Does PMP model map to waterfall model? Does PMP map to agile model? PMI and more specifically the PMBOK doesn’t say you must apply the waterfall approach to a project, nor does it say you can’t apply agile principles to a project. 3. Does lean/Agile have any applicability in the service delivery environment, and how about if your team delivers a combination of services and software development? Should the team work from a consolidated backlog for both activities? Absolutely! Read Michael George’s “Lean Six Sigma for Service”, 2003 McGraw Hill for some key insights into this area 4. My projects consist of a Project Lead (who performs the role of Scrum-Master) and a Project Mgr. In this setup (within Agile) what would be the role of the PM? Actually, the ScrumMaster is the PM. The additional PM is sort of redundant. 5. We are trying to implement Agile in our heavy waterfall environment. How do we balance the compliance/documentation of waterfall against that of Agile? Agile developers are of the mindset that they don't need to do project planning or project documentation? Your developers need training in a disciplined agile project method. Unless you are in an XP environment in which the focus is to get product out the door ASAP, they are incorrect on both counts. Yes you do a plan, but the plan is not a straitjacket. It is a basis for crucial conversations with your customer to deliver the highest value features early and as the customer has defined them. Documentation becomes the byproduct of customer-developer interaction

6. What are the actual "deliverables" for a project? Specifically a BRD, a tech design spec? If you are building software for a customer, then the ultimate deliverable is working software that meets the customer’s fitness-for-use criteria. Then ask your customer what other key deliverables are important from their perspective. 7. What does the SCRUM acronym stand for? The term is taken from Rugby 8. Which two add customer value? Winston Royce wrote that in the waterfall model, the customer finds value in only Analysis (where their business needs are captured) and coding (where they see the work being done). The other activities prior to deployment are considered NVA (non-valued activity) to the customer (!) 9. The scrum methodology assumes that the client (product owner) is able to be fully engaged and available to participate in the creation of the product backlog, sprint planning & reviews. How can you effectively implement scrum when they are not available? One of the most frequent complaints I hear from management is that they have no visibility into the software/IT process. The SCRUM process effectively provides this visibility via the Product Owner. I would first ask the business how important the project is to the business – this will tell you where the project sits in terms of the business’ hierarchy of needs. If it is low visibility, you may have difficulty engaging the Product Owner if they are focused on ‘higher value’ projects. Ask why the project is being implemented. From an ongoing engagement perspective, since the daily stand-up is only 15 minutes a day, I would challenge the stakeholder with this information and ask why they can’t spare a mere 15 minutes a day to gain the valuable visibility into the project that they need. 10. Can you elaborate on how needs are different from requirements? This may be something of a shock but here goes: Business users are not qualified to give the technical team requirements(!) However they can describe in plain English what their business need is (Okay – so you can call this a ‘business requirement’). It is critical that they do so in their own language, and in this process it is your job to keep them from designing the system in front of you (that specifies an instantiation of a solution and you can thank the user for their ideas, but bring them back to the ‘need’ focus. Based on the business needs, the technical team can then create the technical requirements that will meet the business needs of the customer.

11. Rolling wave integration testing sees to be additive over a number of sprints. Considering that a sprint is for a fixed period of time, how is the addition of previously exercised tests fit into this fixed time period? You need automated testing tools that will handle regression testing as well as the rolling wave integration of new elements from the current sprint. Jeff Sutherland (co-creator of SCRUM) is the CTO of a company called PatientKeeper. Over 60+ developers work in simultaneous SCRUMS and deliver 45 software releases a year across about 5 different products (!) 12. How would an organization move from a more traditional SDLC methodology to agile? Triple constraints are measurable. How does an organization attempt to provide value? How is value calculated? Ask the customer what they value and if they are getting that now from your current process. “Value” is an intangible asset. Edwards Deming warned against running a business based on the visible figures alone – they don’t tell you enough. Triple constraints measure budget, timeline and scope – they are the visible figures. Ask your customer if satisfaction of those three elements automatically equates to customer satisfaction. (Does it?) 13. Agile appears to be an excellent way to a quality product. How does the iterative process affect the project scope and project completion date? It appears this methodology opens the door for scope creep and a more expensive project more so than Waterfall. The idea of ‘scope creep’ doesn’t really compute in an agile/SCRUM environment, because the business is focused on identifying and receiving delivery of its highest priority features and needs first. This usually constitutes only the top 20% of the entire feature set the business has requested. The Standish Group collected empirical data on thousands of projects and discovered that with custom software development projects run under a fixed price contract vehicle, 45% of the ‘required’, developed features were NEVER used by the customer. Ask your CFO if it’s OK for you to blow 45% of your budget on developing features the end-user will never use and watch his/her reaction… On the contrary, it appears that it is not the agile approach but the waterfall approach that has actually institutionalized scope creep by having it masquerade as ‘requirements capture’. 14. This method seems like it would create a very difficult environment to control scope and customer expectations on budget and time of delivery. Comments? See# 13

15. What does TPS stand for? Toyota Production System 16. My management believes Agile = faster and cheaper. Can you comment? Not necessarily. Ultimately that can occur, if the business is willing to do the basic blocking-and-tackling necessary for a lean-agile implementation. What the business needs to understand is that the lean-agile process exposes bad wiring in the business very quickly – not only in the IT area but particularly in the business itself. This may be painful for some in the organization – especially execs. It surfaces accountability issues and bad policy. 17. At what stage in the project should the product owner be included in the stand up meetings? From Iteration # 1 18. What do you do to get worker members attend the daily meetings? We're struggling with various members who don't regularly attend. The daily stand-up is where the team gets to status each other on where they are at in their work and what their obstacles are. 19. How important is it to have a fixed Sprint length? I have a Product Owner that wants the Sprint length to change within each iteration. Bad idea. The whole point in keeping sprints at an even duration is that you need a sizing mechanism to determine the team’s velocity so that you can ultimately predict the projects real endpoint with much higher accuracy. With a fixed sprint length, and each team member working the same number of hours per week, you will see how much work actually gets done in the sprint i.e. you are plotting true resource utilization. After about 3 sprints (based on this sizing mechanism) you can extrapolate the actual completion date much more accurately than via conventional methods 20. Who completed and published the productivity survey? Managing New Product and Process Development: Text and Cases, Clark and Wheelwright,1993, Free Press 21. Must the "product owner" be a business person? Can an IT BA who works closely with the business person be the product owner? Only if the BA has been empowered with the authority to prioritize the Product Backlog by the business

22. Do you suggest any software or templates for agile project management? I’d look at VersionOne first and see if it meets your needs. 23. Much of the Lean Six Sigma indoctrinations involve metrics and measurement. For an experienced Agile team, which metrics would you recommend tracking regularly, and how would you integrate these into the Agile process? Start with the customer and ask what is important to them. Then go after internal improvements starting with the five key lean elements of inventory, extra motion, repetitive processing, delay, and unnecessary transportation and how these elements slow down your development process 24. The presentation was focused on Software development, but I wanted to know if SCRUM or Lean PM is applicable to Process Development projects as well. Absolutely. SCRUM is a de facto ‘pull’ system when you think about it. It pulls value through the system based upon the prioritized needs of the customer 25. I was interrupted when you were on the slide where you mentioned the two phases of the traditional Waterfall approach that a customer considers to add value. Can you tell me what they are please? (See the Question # 8 response) 26. I would like to know is there any Software Product or Services companies successfully implemented Agile Models? I would like to get more information and statistical data if possible. Go to http://www.patientkeeper.com . Jeff Sutherland, the co-creator of SCRUM, is running over 60 developers in simultaneous SCRUM cycles. Check out the company’s website and then go to www.jeffsutherland.com (Jeff’s blog) for more information.

Documents

Lean - Agile Project Management - ASPE â€“ SDLC