Six Sigma and CMMI :Showdown in Software Process Engineering

Jonathan Beckham

Process Improvement Goals• Process Goals

– Improve schedule and budget predictability– Improve cycle time / time to market– Increase productivity– Improve quality (as measured by defects)– Increase customer satisfaction– Improve employee morale– Decrease cost of quality

Process Improvement Goals

• Companies want to increase Return On Investment

What affects software?

People

TechnologyProcess

Six Sigma: What is it?

• According to David Card, (IEEE Software 2000), Six sigma is “A generic quantitative approach to improvement that applies to any process.”

• “Six Sigma is a disciplined, data-driven approach and methodology for eliminating defects (driving towards six standard deviations between the mean and the nearest specification limit) in any process -- from manufacturing to transactional and from product to service.” –isixsigma.com

What’s the missing piece?

• In essence, six sigma is an approach to finding the cause of business problems and solving them making an impact on the bottom line of a corporation and decreasing variation for products.

• Six Sigma is more of a methodology or a philosophy.

Six Sigma Levels

• Six Sigma can be identified at three levels: – Metric– Methodology– Philosophy

Metric

• 3.4 Defects Per Million Opportunities (DPMO)

• DPMO allows you take complexity of product/process into account

• Refers to statistics. The +-6 standard deviations (sigmas) away from the standard mean

• Most companies are within 3 standard deviations of the mean

Metric

• The higher the sigma value, the better the process is capable of producing defect free results.

• To be in this six sigma range you must have a quality product 99.9999998% of the time.

Philosophy

• Reduce variation in your business and make customer-focused, data driven decisions

• Translate customer needs into operationalmeasurements

Methodology

• Structured problem solving techniques and roadmap

• Two primary sub-methodologies in Six Sigma– DMAIC – DMADV

DMAIC

• Define-Measure-Analyze-Improve-Control

• Tool for incremental process improvement of existing processes within an organization that fall below required specification

DMAIC cont.• Define

– Define the problem or process to improve upon related to the customer and goals

• Measure– How can you measure this process in a systematic way?

• Analyze– Analyze the process or problem and identify the way in which it can be

improved. What are the root causes of problems within the process?• Improve

– Once you know the causes of the problems, present solutions for them and implement them

• Control– Utilize Statistical Process Control to continuously measure your results

and ensure you are improving– Several Software Packages available to assist in measuring yield, defects

per million opportunities, etc.

DMADV

• Define-Measure-Analyze-Design-Verify• Methodology for producing new processes

that meet the Six Sigma Quality levels desired

• Similar to DMAIC, however, we have a design stage here

• DFSS – Design-For-Six-Sigma– Using models or prototypes to create designs

and ensure they are effective in meeting goals

DMAIC VS. DMADV

Six Sigma Personnel• Champion

– Manager/Director/Executive who makes sure the resources are in place for a Six Sigma project

• Master Black Belts– Quality experts in an organization– Responsible for strategic implementation– Teach/Mentor other Black and Green Belts

• Black Belts– Six Sigma team leaders responsible for implementing process

improvement projects within the business

• Green Belts– Employee of an organization that has some training in Six Sigma

and may lead a Six Sigma project, but only as part of their job

CMM: What is it?

• “A Capability Maturity Model (CMM) is a reference model of mature practices in a specified discipline, used to improve and appraise a group’s capability to perform that discipline” -SEI

• CMM contains a list of key process areas that includes software engineering, project management, and process improvement divided into different levels

CMMI• CMM sunset

– No more SEI CMM training since 2003• Some vendors may still offer it

– CMM Appraisals will end this year• SEI states other vendors may offer this but questions value

• Version 1.1 CMMI released in December 2001– Adds requirements development, validation– Identifies strategy for verification and validation or

products (testing)– Adds relevant stakeholders and level of involvement

(Identify key people)– Less focus on “documented procedures” (Only when

critical to a process)

CMMI Models

• Capability Maturity Model Integration brings together four disciplines– Systems Engineering (SE)– Software Engineering (SW)– Integrated Product and Process Development

(IPPD) (Level 3 add on)– Supplier Sourcing (SS) or Acquisition (Level 3

add on)

CMMI Maturity Levels

Maturity Levels of Organizations 9/28/04

SW-CMMI Strengths

• Engineering focused which assists in integration and interaction among engineering departments

• Recognized Internationally

• DOD contracts require certain levels for bids

• Utilizes best practices in the industry

Six Sigma Strengths

• Generic Process Improvement Strategy can be applied to any process

• Trained personnel working on process improvements for organization on continuous basis

SW-CMMI Weaknesses• Statistical methods often not used at levels 4 and

5 when they could ease data analysis– Six Sigma embraces statistical methods– Improvement in CMMI

• CMMI goals are not always aligned with business and customer goals– Generally Software Specific, but improving with CMMI

• CMMI assessors are certified, but implementers are not so they may not understand underlying concepts (statistical process control, etc)– Six Sigma has competence certifications for relevant

subject matter

Six Sigma Weaknesses• Derived from manufacturing which has a real tangible

process that is easy to measure and apparent, as opposed to software engineering

• No formal connection to ISO 9000 – ISO 9000 and CMM have milestones and certifications

• Six Sigma relies on training personnel (black belts) that look for areas that need improvement which may not be the best way to go about this– Costs also involved in training for Six Sigma

• Focuses on Internal process experience and doesn’t consider external technology or best practices

CMMI Case Studies

• Lockheed Martin M&DS– SW CMM ML 2 (1993) – SW CMM ML 3 (1996) – CMMI SW ML5 (2002)– Increased software productivity by 30%– Decreased unit software cost by 20%– Decreased defect find and fix costs by 15%

CMMI Case Studies

• Bosch Gasoline Systems– CMM Level 4– Predictability – internal on-time delivery

improved by 15%– Less Rework – first pass yield improved by

10%– Product Quality – reduction in error cases in

company by one order of magnitude

Six Sigma Case Study

• >$3 billion energy company

• Customer dissatisfaction with software deliverables

• Customer satisfaction improved by 2 points on their “report card”

• Product quality was improved by 30%

• Reduction of rework by 50% was achieved

• Project savings of $150 K was achieved

So which one is better?

• “The is no silver bullet” – Fred Brooks

• Neither one is necessarily “better”

• Both have weaknesses

• Both have multiple case studies and large corporations that have realized real savings and process improvements utilizing their techniques

• Utilizing both is possible

CMM and Six Sigma Similarities

• Both emphasize reducing defects as their main process improvement goal

• Both focus on quantitative decision making that relies on measurement– Six Sigma and CMM level 4 and 5

• Both utilize similar basic statistical process control techniques – As mentioned, Six Sigma has a better

foundation in this area

Using Six Sigma and CMM

• Six sigma provides a foundation to define, measure, analyze, improve, and control the process– Utilize process mapping

• Can work in conjunction with CMM to map out a process and identify gaps within software processes

– Failure Modes and Effects Analysis• Detects how a process can fail and assigns severity

to it to prioritize actions

Using Six Sigma and CMM

• CMM level 4 and 5 and Six Sigma are often seen as similar:– Six Sigma training can help to deal with CMM’s

Quantitative Process Management and Defect Prevention requirements

– CMM can clarify how to use Six Sigma techniques for software process

– Important to note that organizations can achieve level 4 and 5 without using Six Sigma techniques

• Many utilize their own statistical methods and analysis• Sometimes ad-hoc, as mentioned before CMM does not tell

you necessarily how to accomplish a level, just what need to be accomplished

Benefits fromSW CMM and Six Sigma

• Study of utilizing them together:– Tata Consultancy Service development center reduced cost of

process failures from 5 to 1 percent, thus reducing total quality cost

– Improved cycle time and productivity– Scheduled slippage was reduced from a range of +-20 percent to

+-2 percent saving the company roughly $700k over 3 years– Amazingly, at the time of the article, clients had not reported

any defects in their software products!

Summary

• Six Sigma is a process methodology that focuses on quantitatively measuring a process in order to control and improve upon it

• Six Sigma strives for 3.4 Defects Per Million Opportunities (DPMO) so that it is within six standard deviations of the mean

• SW CMMI is a process improvement model for software engineering that is divided into levels and contains a list of key process areas

Summary

• Six Sigma and CMMI can be utilized together to improve the software engineering process

• Six Sigma’s more advanced statistical methods can be applied to assist in quantitatively analyzing data within CMMI’s level 4 and 5

• Six Sigma can also complement any individual process within CMMI by utilizing techniques such as process mapping

Conclusion

• Lastly, the effects of implementing Six Sigma and CMMI can highly improve the quality of the product developed and individual processes which can directly impact a business's bottom line and customer satisfaction

References• David N. Card, “Sorting out Six Sigma and the CMM”, IEEE Software,

May June 2000, pp. 11-13 • Jack Horgan, “Six Sigma and CMM”, September 13 - 17, 2004,

http://www10.edacafe.com/nbc/articles/view_weekly.php?articleid=209200

• M. Murugappan and G. Keeni, “Blending CMM and Six Sigma to Meet Business Goals”, IEEE Software, March/April 2003

• “IT Perspective: Balancing Six Sigma and the Capability Maturity Model (CMM ®/CMMI SM )”, http://www.gartner.com/4_decision_tools/measurement/measure_it_articles/2002_10/six_sig.jsp

• SEI Website, http://www.sei.cmm.edu/• Six Sigma, http://www.isixsigma.com/• Thomas Pyzdek, "DMAIC and DMADV", Pyzdek Consulting, Inc. Six

Sigma Handbook, http://www.pyzdek.com/DMAICDMADV.htm

QuestionsOr

Comments?