MTAT.03.243 Software Engineering Management · 2014-03-24 · MTAT.03.243 / Lecture 10 / © Dietmar Pfahl 2014 MTAT.03.243 Software Engineering Management Lecture 10: Lean Principles

MTAT.03.243 / Lecture 10 / © Dietmar Pfahl 2014

MTAT.03.243

Software Engineering Management

Lecture 10:

Lean Principles and

Processes

Dietmar Pfahl

email: [email protected] Spring 2014


Structure of Lecture 9

• Background and Motivation

• Lean Principles

• Types of Waste

• Removing Waste

– Case Study

• Tools for Lean Development

• Wrap-Up


There exists more than one Agile Method!

Crystal

Clear

AUP

DSDM

FDD

Scrum

XP

AUP = Agile Unified Process

DSDM = Dynamic Systems Development Method

FDD = Feature-Driven Development

XP = Extreme Programming

…

KANBAN

LEAN

…


Origins of

Lean Software Development

• Originates from Toyota Production System (TPS)

Also called Just-In-Time system

• Post WWII Japanese automobile industry could not compete with U.S.

mass production systems

• Inspiration for TPS found in the 1950’s from U.S. supermarkets

Customers could get what they wanted, when they wanted it and

shelves were refilled when items were about to run out.

• The concepts transferred to the domain of software engineering by

Mary and Tom Poppendieck (2003, 2007).


Main Goals of LEAN

1. All processes shall give value

– Remove everything that does not create value

2. Ensure good flow in the processes to avoid bottlenecks and queues (-> work piling up and waiting)

3. All activity shall be based on need (-> Pull)

– If there is no demand for a product or service, the related task is unneccesary

4. Become a learning organization with focus on continuous stepwise improvement

– Kaizen (= small change for the better)


Focus on reducing the activities that do not

create value

LEAN approach

to continuous

improvement

Focus on

removing/reducing

the activities that do

not create value for

our customers

Traditional

approach

Focus on the

efficiency of the

activities that create

value for the

customers



Types of Work

Urgent Not urgent

Important

Not important


Value Stream Mapping

• (A form of process modelling)

• Helps us understand our processes and which activities

add value and which do not




• Lean Principles

• Types of Waste

• Removing Waste

– Case Study


• Wrap-Up


Seven Principles of Lean SW Development (by Mary Poppendieck)

1.Optimize the Whole

2.Eliminate Waste

3.Build Quality In

4.Learn Constantly

5.Deliver Fast

6.Engage Everyone

7.Keep Getting Better




2.Eliminate Waste

3.Build Quality In

4.Learn Constantly

5.Deliver Fast

6.Engage Everyone


Optimizing a part of a system will

always, over time, sub-optimize the

overall system.

– Focus on the Entire Value Stream

From concept to cash.

From customer request to deployed

software.

– Deliver a Complete Product

Customers don't want software; they want

their problems solved. Complete solutions

are built by complete teams.

– Think Long Term

Beware of governance and incentive

systems that drive short term thinking and

optimize local performance.




2.Eliminate Waste

3.Build Quality In

4.Learn Constantly

5.Deliver Fast

6.Engage Everyone


The three biggest wastes in software

development are:

– Building the Wrong Thing

"There is nothing so useless as doing

efficiently that which should not be done at

all." – Peter Drucker

– Failure to Learn

Many of our policies – for example:

governance by variance from plan, frequent

handovers, and separating decision-making

from work – interfere with the learning that

is the essence of development.

– Thrashing

Practices that interfere with the smooth flow

of value – task switching, long lists of

requests, big piles of partly done work –

deliver half the value for twice the effort.




2.Eliminate Waste

3.Build Quality In

4.Learn Constantly

5.Deliver Fast

6.Engage Everyone


If you routinely find defects in your

validation process, your process is

defective.

– Final Validation Should Not Find Defects!

Every software development process ever

invented had as its primary purpose to find

and fix defects as early in the development

process as possible.

– Mistake-Proof your Process with Test-First

Development

Tests – including, unit tests, end-to-end tests,

and integration tests – must be available to

establish confidence in the correctness of the

system at any time during development, at

every level of the system.

– Break Dependencies

System architecture should support the

addition of any feature at any time.




2.Eliminate Waste

3.Build Quality In

4.Learn Constantly

5.Deliver Fast

6.Engage Everyone


Planning is useful. Learning is essential.

– Predictable Performance is Driven by

Feedback

A predictable organization does not guess

about the future and call it a plan; it develops

the capacity to rapidly respond to the future as

it unfolds.

– Maintain Options

Think of code as an experiment – make it

change-tolerant.

– Last Responsible Moment

Learn as much as possible before making

irreversible decisions. Don't make decisions

that will be expensive to change before their

time – and don't make them after their time!




2.Eliminate Waste

3.Build Quality In

4.Learn Constantly

5.Deliver Fast

6.Engage Everyone


Start with a deep understanding of all

stakeholders and what they will value. Create a

steady, even flow of work, pulled from this

deep understanding of value.

– Rapid Delivery, High Quality, and Low Cost are

Fully Compatible

Companies that compete on the basis of speed have

a big cost advantage, deliver superior quality, and are

more attuned to their customers' needs.

– Queuing Theory Applies to Development, not Just

Servers

Focusing on utilization creates a traffic jam that

actually reduces utilization. Drive down cycle time with

small batches and fewer things-in-process.

Aggressively limit the size of lists and queues

– Managing Workflow is a lot easier than Managing

Schedules

The best way to establish reliable, predictable

deliveries is to establish reliable, repeatable workflows

with iterations or a kanban system.




2.Eliminate Waste

3.Build Quality In

4.Learn Constantly

5.Deliver Fast

6.Engage Everyone


The time and energy of bright, creative people

are the scarce resources in today's economy,

and the basis of competitive advantage.

People who are paid fairly and adequately are

motivated by autonomy, mastery, and purpose.

– Autonomy

The most effective work groups are semi-autonomous

teams with an internal leader with end-to-end

responsibility for complete, meaningful tasks.

– Mastery

Respect for people means providing the challenge,

feedback, and environment that enables everyone to

become excellent.

– Purpose

Tie work to value. Only by believing in the purpose of

their work will people become engaged in achieving

that purpose.




2.Eliminate Waste

3.Build Quality In

4.Learn Constantly

5.Deliver Fast

6.Engage Everyone


Results are not the point – the point is to

develop the people and the systems capable of

delivering results.

– Failure is a Learning Opportunity

The most reliable performance comes when

even small failures are deeply investigated

and corrected; when noise is not tolerated.

– Standards Exist to be Challenged and

Improved

Embody the current best known practice in

standards that everyone follows, while actively

encouraging everyone to challenge and

change the standards.

– Use the Scientific Method

Teach teams to: establish hypotheses,

conduct many rapid experiments, create

concise documentation, and implement the

best alternative.




• Lean Principles

• Types of Waste

• Removing Waste

– Case Study


• Wrap-Up




2.Eliminate Waste

3.Build Quality In

4.Learn Constantly

5.Deliver Fast

6.Engage Everyone



Seven Wastes of Software Development

• Handoffs. Passing the information/work to someone else, getting

information/work from someone else.

• Partially done work. Something that is not done. E.g. untested code,

undocumented or not maintained code.

• Task switching. How many other tasks people need to do. E.g. the amount of

projects done simultaneously.

• Delays. Waiting for something.

• Extra features. Something that is not really needed.

• Defects. Something that does not meet the targets, or is not what it is

supposed to be. E.g. software bugs, incorrectly implemented business

requirements.

• Relearning (waste of knowledge). E.g. forgetting decisions, re-trying

solutions already tried, the inability to utilize the knowledge of other people.


Waste in Software Development

Handoffs


Example of Over-

processing and

Handoffs

• Defect handling

process in globally

distributed

development

– Spec: Germany

– Dev: India

– Test: Austria

>20 process steps

Tasks:

• New FR

• Completion (CMP)

• Pre-Analysis (PRE)

• Analysis (ANA)

• Correction (COR)

• Verification (VRF)

• Closing (CLS)


Example of heavy design – Four projects

developing the same system

Much effort on

functional design had

a negative effect on

project and (partly)

product quality

Project management

Interaction design

Functional design

Development Test

0

100

200

300

400

500

600

Source: Anda, B.C.D.; Sjoberg, D.I.K.; Mockus, A., "Variability and Reproducibility in Software Engineering: A Study of Four Companies that Developed the Same System," IEEE Transactions on Software Engineering , vol.35, no.3, pp.407,429, May-June 2009 doi: 10.1109/TSE.2008.89

90 87 79

A B C D




A B

D C

Example of Amount of Defect Correction

Q: How do the processes differ?






• Lean Principles

• Types of Waste

• Removing Waste

– Case Study


• Wrap-Up


Value Stream Mapping

• Map and visualize each individual step, e.g., in product

development from customer request to completed product.

• Identify the steps and actions that create customer value and

the steps and actions that can be considered as waste.

• Plan actions for removing the non-imposed waste.

• Even though the focus is on ”optimizing the whole”, Value

Stream Mapping can be applied to sub-processes, too.


Value Stream Mapping Process

Three general steps (Abdulmalek and Rajkopal):

1. Choose a particular product or product family as the target for improvement.

2. Draw a current state map of the process.

– This can be seen as a snapshot of how things are currently being done and is created by “walking along” the process.

– This provides the basis for analyzing the system and identifying its weaknesses.

3. Create a future state map.

– This is a picture that depicts how the system should look like when wastes have been removed.

• The weaknesses of the process can be further elaborated for example by applying the technique of “Five Why’s” which aims to identify the root-cause behind the weaknesses.

F. A. Abdulmalek and J. Rajgopal. Analyzing the benefits of lean manufacturing and value stream mapping via simulation: A process sector case study. Int. J. Production Economics 107(1), 2007. pp. 223-236.


The Five Why’s

• Method to identify root-

causes of problems

• The purpose is to fix the

system, not just remove

the symptom.

– If we aren’t clear

about the difference

between symptoms

and problems, we will

not find the root cause

effectively.

• Example with the problem that a key piece

of equipment failed:

1. Why did the equipment fail? Because

the circuit board burned out.

2. Why did the circuit board burn out?

Because it overheated.

3. Why did it overheat? Because it

wasn’t getting enough air.

4. Why was it not getting enough air?

Because the filter wasn’t changed.

5. Why was the filter not changed?

Because there was no preventive

maintenance schedule to do so.

That is now a root cause that can be

solved. By focusing on the question

WHY, we are more likely to avoid using

the other W question: WHO.


Example 1 of Value Stream Mapping based on a

real-world industry case

• Large software

intensive company

using an agile

process.

• Group of people

responsible for

gathering and

managing software

requirements

before they enter

development

stage.

Sales people

Marketing

Requirements’

sources

Entering reqs. to the

systemAnalysis

Management

Acceptance

Prioritization Release Planning

VAT: 1 hourVAT: 3 hour

VAT: 1 hour

VAT: 1 hour

VAT: 1 hour

NVAT: 5 days

NVAT: 3 days

NVAT: 4 days

NVAT: 2 days

VALUE ADDING TIME (VAT): 7 hours

NON-VALUE ADDING TIME (NVAT): 14 days Development process


Future State Value Stream Map

Sales people

Marketing

Requirements’

sources


systemAnalysis

Management

Acceptance



VAT: 1 hour

VAT: 1 hour

VAT: 1 hour

NVAT: 3 days

NVAT: 2 days

NVAT: 3 days

NVAT: 2 days


NON-VALUE ADDING TIME (NVAT): 10 daysDevelopment process

Problem: Requirements were often incomplete and thus had to be reworked before analysis could be done Improvement: Better guidance on how to capture and enter in the system complete requirements Saving: 2 days



Sales people

Marketing

Requirements’

sources


systemAnalysis

Management

Acceptance



VAT: 1 hour

VAT: 1 hour

VAT: 1 hour

NVAT: 3 days

NVAT: 2 days

NVAT: 3 days

NVAT: 2 days



Problem: Management Acceptance is a Type One Muda – thus cannot be eliminated Waiting for managment to make accept decision Improvement: Better mechanism to make management aware that requirements are waiting for acceptance Saving: 1 day



Sales people

Marketing

Requirements’

sources


systemAnalysis

Management

Acceptance



VAT: 1 hour

VAT: 1 hour

VAT: 1 hour

NVAT: 3 days

NVAT: 2 days

NVAT: 3 days

NVAT: 2 days



Problem: Waiting time for somebody to do the prioritization Improvement: Clearer definition of responsibilities and assignment of roles Saving: 1 day


Notations for Value Stream Maps

• Many different

notations exist

– one example

is shown on

the right:




• Lean Principles

• Types of Waste

• Removing Waste

– Case Study


• Wrap-Up


Value Stream Analysis – Application Example

Research Goal:

• Find out how empirical methods

can help improve the software

testing process(es) in a Swedish

Automotive SW Organisation

Reference:

• Abhinaya Kasoju, Kai Petersen, Mika V.

Mäntylä, Analyzing an automotive testing

process with evidence-based software

engineering, Information and Software

Technology, Available online 8 February

2013, ISSN 0950-5849,

10.1016/j.infsof.2013.01.005.

Research Design:

EBSE = Evicence-Based Software Engineering



Case Study:

To identify current Strengths and

Challenges/Issues

How:

• Analyze existing Process

Documentation

• Conduct Interviews

– 14 Persons

– 7 different roles

Where:

• 3 Departments

• 8 Projects

Research Design:




Case Study: List of projects analyzed



Case Study:


Challenges/Issues

How:


Documentation


– 14 Persons


Where:

• 3 Departments

• 8 Projects

Research Design:



Value Stream Analysis

– Application Example

Case Study:

Testing process (unifying the

information received from all 8

projects)




Case Study:



projects)

Test Planning:

Estimate the requirements, test

techniques, tools and other test

artifacts

Test scheduling




Case Study:



projects)

Test Analysis

and Design:

Update test plans

Identify and design test scripts and

test data




Case Study:



projects)

Test Build:

Collect and build all the required

test environment, test scripts, and

other test data designed during the

previous stage




Case Study:



projects)

Test Execution

and Reporting:

Run tests and record defects

Evaluate test results and generate

a report



Case Study:


Challenges/Issues

How:


Documentation


– 14 Persons


Where:

• 3 Departments

• 8 Projects

Research Design:




Case Study: Strengths and Good Practices

Vary with team size:

Most of the practices considered as strengths in small teams were

not considered strengths in large teams and vice versa.

• Work in small, agile teams

• Communication

• Shared roles and responsibilities

• Test techniques, tools, and environments

Further refined into 12 strength items • 3 for small teams • 6 for large teams • 3 for both small and

large teams



Case Study:


Challenges/Issues

How:


Documentation


– 14 Persons


Where:

• 3 Departments

• 8 Projects

Research Design:




Case Study: Challenges/Issues

• C01: Organisation of testing and

its process

• C02: Time and cost constraints for

testing

• C03: Requirements

• C04: Resource constraints for

testing

• C05: Knowledge management

• C06: Interactions and

communication for testing

• C07: Testing techniques, tools

and environments

• C08: Quality aspects and their

incorporation in testing

• C09: Defects detection

• C10: Documentation of testing

10 Challenge Areas identified – Issues related to:





its process

• C02: Time and cost constraints for

testing



testing





and environments





C01

C02

C03

C04

C05





its process

• C02: Time and cost constraints

ftesting



testing





and environments





C06

C07

C08

C09

C10



Case Study:

Identify value-adding activities

and waste (waiting time / non-

value-adding)

• Value-adding is, e.g., assuring

the quality of a product

Value Stream Mapping Notation:

Research Design:





Case Study: Value




Case Study: Waste

7


Case Study:

Current State Map



Test Planning

Test Analysis &

Design

Test Build

Test Execution

& Reporting


Case Study: Current State Map


Test Planning

• C01: Organisation of testing

and its process



testing

(3) W4, W5: C04

(3) W2, W3: C03

• W01: Partially done work

• W02: Extra features

• W03: Relearning

• W04: Handoffs

• W05: Task switching

(3) W4, W5: C04

(2) W2, W3,

W4: C03

(1) W1: C01, C03




Test Planning

• C01: Organisation of testing

and its process


• W01: Partially done work

“The waste observed in sub-process area 1 is partially done work (W01). The reason for

not completing tests are the lack of planning tests due to lack of test definition and testing

done in haste (C01), ultimately resulting in conducting tests in an unstructured manner

with low test coverage. This is amplified by unclear requirements (C03).”

(3) W4, W5: C04

(3) W2, W3: C03

(3) W4, W5: C04

(2) W2, W3,

W4: C03

(1) W1: C01, C03




Test Planning


• W02: Extra features

• W03: Relearning

• W04: Handoffs

“We identified the wastes “extra features” (W02) and “handoffs” (W04). Extra features that

are at times removed from the system prior to release, even though they were

implemented, e.g. due to volatile and unclear requirements (C03). However, testing is

also performed on such features/functions. This waste occurs in the form of effort that is

put in writing the test plan and subsequently scheduling tests and allocating resources.

Unclear requirements (C03) further require “relearning” (W03).”

(3) W4, W5: C04

(3) W2, W3: C03

(3) W4, W5: C04

(2) W2, W3,

W4: C03

(1) W1: C01, C03




Test Planning


testing

(3) W4, W5: C04

(3) W2, W3: C03

• W04: Handoffs

• W05: Task switching

(3) W4, W5: C04

(2) W2, W3,

W4: C03

(1) W1: C01, C03

“One general issue in the case organization is resource constraints (C04). The wastes

that occur here are lack of availability of testers (W04: Handoffs) and unclear roles and

responsibilities as a part of organization structure, which hinders the formation of right

teams, resulting in task switching (W05).”


Case Study:

Current State Map



Test Planning

Test Analysis &

Design

Test Build

Test Execution

& Reporting

done



Case Study:


Challenges/Issues

How:


Documentation


– 14 Persons


Where:

• 3 Departments

• 8 Projects

Research Design:




Case Study: Systematic

Literature Review

• Searched in digital libraries:

– ScienceDirect

– ACM Digital Library

– WileyInterscience

– SpringerLink

– IEEE Xplore

• Created 5 search strings

• Found 707 papers of which 48

were relevant

– Only papers after year 2000 and in Englsh

were considered

Solution Proposals (SPs) found:

• SP1: Requirements Management (RM)

• SP2: Competence Management (CM)

• SP3: Quality Assurance and Standards

• SP4: Test Automation

• SP5: Test Tool Deployment

• SP6: Agile Incorporation

• SP7: Test Management

– Test process management

– Test artifacts and assets organization

– Requirements management in accordance

with testing

– Competence management

– Defect management



Case Study:


Challenges/Issues

How:


Documentation


– 14 Persons


Where:

• 3 Departments

• 8 Projects

Research Design:



Case Study:

Current State Map



Future State Map (showing 1 Iteration)

Only proposal – not yet evaluated)




• Lean Principles

• Types of Waste

• Removing Waste

– Case Study


• Wrap-Up


Some Lean Tools

• Value stream mapping

–Identify sources of waste and problem areas

• Problem solving and A3

–Problem identification and root-cause analysis

• Measures and visual indicator

–Information radiator of status

• Open-space meeting

• Stop-doing list

– Create time for improvement


A3 Thinking

Tool to:

• get people to think methodically and take initiative

• think like scientists

– State hypothesis

– Design experiment

– Test

– Assess results and plan next step based on evidence

• and learn by doing

• Link: http://www.coe.montana.edu/IE/faculty/sobek/A3/index.htm

http://www.coe.montana.edu/IE/faculty/sobek/A3/index.htm


A3

Report


Open-space meeting

• Meetings with a specific format

where the participants

– bring their own topics,

– form groups based on

interest,

– discuss for a limited time,

and

– bring main result back to

plenum




• Lean Principles

• Types of Waste

• Removing Waste

– Case Study


• Wrap-Up


Recap

• Lean development is

focused on removing

waste from a system

and improving value for

the customer

• Principles:

– Optimize the Whole

– Eliminate Waste

– Build Quality In

– Learn Constantly

– Deliver Fast

– Engage Everyone

– Keep Getting Better


Comparison of Lean vs. Agile Principles

• Optimize the whole

– All agile principles

• Eliminate waste

– Simplicity is essential

– Satisfy customer through early and continuous delivery

– Working software is the primary measure of progress

• Build quality in

– Working software is the primary measure of progress

• Learn constantly

– Welcome changes

• Deliver fast

– Deliver fast and frequently

– Satisfy customer through early and continuous delivery

• Engage Everyone

– Self-Organizing Teams

• Keep getting better

– Regular reflection

– Close collaboration


Lean vs. process improvement

frameworks

• Must consider people and culture not only

processes

• Top-down approaches to change and improvement

seldom work very well

• Using best practices does not create a competitive

edge

• Process improvement and best practices often lead

to administrative overhead, and hence, waste


Lean Management – List of Tips (Liker 2004)

• Base your management decisions on a long-term philosophy, even at the expense of short-term financial goals.

• Create continuous process flow to bring problems to the surface.

• Use “pull” systems to avoid overproduction.

• Level out the workload.

• Build a culture of stopping to fix problems to get quality right the first time.

• Standardized tasks are the foundation for continuous improvement and employee empowerment.

• Use visual control so no problems are hidden.

• Use only reliable, thoroughly tested technology that serves your people and processes.

• Grow leaders who thoroughly understand the work, live the philosophy and teach it to others.

• Develop exceptional people and teams who follow your company’s philosophy.

• Respect your extended network of partners and suppliers by challenging them and helping them improve.

• Go and see for yourself to thoroughly understand the situation.

• Make decisions slowly by consensus, thoroughly considering all options; implement decisions rapidly.

• Become a learning organization through relentless reflection and continuous improvement.


Focus on processes, management and culture

Culture

Manage- ment

Process

Continuous improvement requires focus on processes, management and culture to succeed!

Improving work processes

• Smart processes

• Established levels of quality

Operative management

• Go and see

• Involved and responsible staff

• Optimal use of resources

• Development of skilles

Culture

• Awareness of what affect culture

• Create openness, curiosity, will to change and ambitions to improve

• Focus on what you can actually improve


This wheel should spin..

1. STRUCTURED

PROBLEM SOLVING

2. IMPLEMENT MEASURES

4. ENSURE IMPROVEMENT OR

ADJUST

3. CHECK EFFECT

Identify need for

improvement

KF

Changed behaviour and

work processes


Acknowledgements

Parts of this lecture are based on material from Bente Anda’s

Lecture on Lean Software Development, presented in

September 2012 at the University of Oslo


Literature

• Middleton, Peter, & Sutton, James. (2005). Lean software strategies. Productivity

Press

• Ohno, T. (1988). Toyota Production System: Beyond Large-Scale Production.

Productivity Press.

• Poppendieck, Mary, Poppendieck, Tom. (2003). Agile software development

ecosystems. Addison-Wesley Professional.

• Poppendieck, M and Poppendieck, T. (2007). Implementing Lean Software

Development: From Concept to Cash. Addison-Wesley.

• Womack, J.P., & Jones, D.T. (2003). Lean thinking: banish waste and create wealth

in your corporation, revised and updated. New York: Free Press.

• Womack, James, & Jones, Daniel. (2005). Lean solutions: how companies and

customers can create value and wealth together. New York: Free Press.

• Ødegård, F. (2007). Lean execution: six questions for software executives. Retrieved from

http://www.leansoftwareinstitute.com/Lean%20Execution%20-%20Six%20Questions%20for%20Software%20Executives.pdf


Next Lecture

• Topic:

– Flow-based (KANBAN) Principles and Processes

• For you to do:

– Continue working on Homework 3

– Read the articles related to this lecture (will be posted

on course web)

Documents

MTAT.03.243 Software Engineering Management · 2014-03-24 · MTAT.03.243 / Lecture 10 / © Dietmar Pfahl 2014 MTAT.03.243 Software Engineering Management Lecture 10: Lean Principles