Process Improvement Workshop Process Improvement in the Aerospace Industry Caltech Aerospace Seminar 13 April 2009 Rick Hefner, Ph.D. Director, Process

Process Improvement Workshop

Process Improvement in the

Aerospace Industry

Caltech Aerospace Seminar13 April 2009

Rick Hefner, Ph.D.Director, Process Management

Northrop Grumman Corporation

[email protected]


2

Agenda

• Current Challenges Facing the Aerospace Industry

• Current Industry Approaches– Capability Maturity Model Integrated– Lean Six Sigma– Agile

• Northrop Grumman Approach


3

NDIA Top 5 Systems Engineering Issues (2003)

• Lack of awareness of the importance, value, timing, accountability, and organizational structure of SE on programs

• Adequate, qualified resources are generally not available within Government and industry for allocation on major programs

• Insufficient SE tools and environments to effectively execute SE on programs

• Requirements definition, development and management is not applied consistently and effectively

• Poor initial program formulationhttp://www.ndia.org/Content/ContentGroups/Divisions1/Systems_Engineering/PDFs18/Modeling_Committee_PDFs/February2003_top_5_issues.pdf


4

NDIA Top 5 Systems Engineering Issues (2006)

• Key systems engineering practices known to be effective are not consistently applied across all phases of the program life cycle.

• Insufficient systems engineering is applied early in the program life cycle, compromising the foundation for initial requirements and architecture development.

• Requirements are not always well-managed, including the effective translation from capabilities statements into executable requirements to achieve successful acquisition programs.

• The quantity and quality of systems engineering expertise is insufficient to meet the demands of the government and the defense industry.

• Collaborative environments, including SE tools, are inadequate to effectively execute SE at the joint capability, system of systems (SoS), and system levels.

Systems Engineering Update, NDIA Top 5 Issues Workshop. July 26, 2006. Briefing by Mr. Robert Skalamera


5

Summary - Evolution of Top SE Issues

Lack of awareness of SE importance

Lack of adequate, qualified resources

Insufficient SE tools and environmentsInconsistent requirements definition

Poor initial program formulation

Inconsistent SE practices across all life cycle phases

Insufficient SE early in the life cycle

Requirements not well managed or translated

Insufficient quantity and quality of SE expertiseInadequate tools and collaborative environments

Complex systems, systems of systems

2003 2006


6

Agenda





7

Two Complimentary Approaches to Process Improvement

Data-Driven (e.g., Six Sigma)

• Clarify what your customer wants (Voice of Customer)– Critical to Quality (CTQs)

• Determine what your processes can do (Voice of Process)– Statistical Process Control

• Identify and prioritize improvement opportunities– Causal analysis of data

• Determine where your customers/competitors are going (Voice of Business)– Design for Six Sigma

Model-Driven (e.g., CMMI)

• Determine the industry best practice– Benchmarking, models

• Compare your current practices to the model– Appraisal, education

• Identify and prioritize improvement opportunities– Implementation– Institutionalization

• Look for ways to optimize the processes


8

The Frameworks QuagmireSarah A. Sheard, Software Productivity Consortium

http://stsc.hill.af.mil/crosstalk/1997/sep/frameworks.asp


9

Heritage of Standards for Systems Engineering

EIA / IS 632

ISO/IEC 15288

Mil-Std-499BMil-Std-

499A

1994

1994

1994

1998

2002

1974

(Not Released)Mil-Std-499

1969

(Trial Use)IEEE 1220

1998

(Full Std)

LegendLegendSupersedesSource for

EIA632

1998

EIA/IS 731

SE CM

IEEE 1220

(Full Std)(Interim Standard)

2002

CMMI-

SE/SW/IPPD

(Interim Standard)

(FDIS)

ISO/IEC 19760

2002

(PDTR)

2002

ISO/IEC 15504

(FDIS)

Standards for Systems Engineering, Jerry Lake, 2002


10

EIA/ANSI 632

Draft Report ISO Study Group May 2, 2000

Breadth and Depth of Key SE Standards

System life

ISO/IEC 15288

Le

vel o

f d

etai

l

Conceptualize DevelopTransition to

Operation

Operate,Maintain,

or EnhanceReplace

or Dismantle

Processdescription

High levelpractices

Detailedpractices

• ISO/IEC 15288 - Common framework for describing the life cycle of systems

• EIA/ANSI 632 - Integrated set of fundamental processes to aid a developer in the engineering or re-engineering of a system

• IEEE 1220 - Standard for managing systems engineering

IEE

E 1

220 Input to 632/1220


11

SEI’s Strategic Classification Taxonomy

Summary of the SEI approach of harmonizing multiple models, by Jeannine Siviy and Pat Kirwan, 2008 PrIME Workshop, http://www.sei.cmu.edu/prime/hardquestionsoutput.html


12

Problem/Solution Space

CMMI

Lean Six SigmaAgile

•Benchmark with community; demonstrate process capability to customers

•Learn new practices•Establish infrastructure•Encourage consistency

across projects

• Improve quality, cost, time-to-market

•Prioritize improvements by business value

•Root cause analysis•Statistical analysis• Identify/implement

focused improvements•Reduce waste


13

Agenda





14

What is the Capability Maturity Model Integrated?

• The CMMI is a collection of industry best-practices for engineering, services, acquisition, project management, support, and process management– Developed under the sponsorship of DoD– Consistent with DoD and commercial standards

• CMMI for Development - used by engineering organizations– Version 1.0 released in 2000, v1.1 in Mar 2002, v1.2 (CMMI-DEV) in

Aug 06

• CMMI for Acquisition - used by buyers (e.g., govt. agencies)– Version 1.0 released in 2007

• CMMI for Services - used by service providers (e.g., help desk)– Under development


15

The Basic Building Blocks of CMMI – 22 Process Areas

Project Management– Project Planning– Project

Monitoring and Control

– Supplier Agreement Management

– Integrated Project Management)

– Risk Management– Quantitative

Project Management

Engineering

– Requirements Development

– Requirements Management

– Technical Solution

– Product Integration

– Verification– Validation

Support

– Configuration Management

– Process and Product Quality Assurance

– Measurement and Analysis

– Decision Analysis and Resolution

– Causal Analysis and Resolution

Process Management– Organizational

Process Focus– Organizational

Process Definition

– Organizational Training

– Organizational Process Performance

– Organizational Innovation and Deployment

Implemented byeach project

Implemented bythe organization


16

Expected Practices Provide Guidancefor Implementation & Institutionalization

SG 1 Establish Estimates SP 1.1 Estimate the Scope of the

ProjectSP 1.2 Establish Estimates of Work

Product and Task AttributesSP 1.3 Define Project Life CycleSP 1.4 Determine Estimates of Effort

and CostSG 2 Develop a Project Plan

SP 2.1 Establish the Budget and Schedule

SP 2.2 Identify Project RisksSP 2.3 Plan for Data ManagementSP 2.4 Plan for Project ResourcesSP 2.5 Plan for Needed Knowledge and

SkillsSP 2.6 Plan Stakeholder InvolvementSP 2.7 Establish the Project Plan

SG 3 Obtain Commitment to the PlanSP 3.1 Review Plans that Affect the

ProjectSP 3.2 Reconcile Work and Resource

LevelsSP 3.3 Obtain Plan Commitment

GG 2 Institutionalize a Managed Process GP 2.1 Establish an Organizational

PolicyGP 2.2 Plan the ProcessGP 2.3 Provide ResourcesGP 2.4 Assign ResponsibilityGP 2.5 Train PeopleGP 2.6 Manage ConfigurationsGP 2.7 Identify and Involve Relevant

StakeholdersGP 2.8 Monitor and Control the ProcessGP 2.9 Objectively Evaluate AdherenceGP 2.10 Review Status with Higher

Level ManagementGG 3 Institutionalize a Defined Process

GP 3.1 Establish a Defined ProcessGP 3.2 Collect Improvement Information

Project Planning - Implementation Project Planning - Institutionalization


17

Practice Ratings for the Organization/Projects


18

How is the CMMI Used for Process Improvement?

www.sei.cmu.edu/ideal/

IDEAL Model


19

Organizational Infrastructure Required for CMMI Level 3

Policies, Processes,Templates & Tools

Best-Practice Libraries

Process Group

Audits & AppraisalsMeasurement RepositoriesPredictive Modeling

Developing and maintaining mature processes requires significant time and investment in infrastructure

Developing and maintaining mature processes requires significant time and investment in infrastructure

Process ImprovementTraining Program

Communications

0

5

10

15

20

25

1 11 21 31 41 51 61 71

UCL

_X

Defe

cts

per

com

ponent

Component #

0

5

10

15

20

25

1 11 21 31 41 51 61 71

UCL

_X

Defe

cts

per

com

ponent

Component #


20

Benefits

• The typical benefits are:– Reduced cost– Faster schedules– Greater productivity– Higher quality– Increased customer satisfaction

• Benefits are cited in many forums– DoD DACS website: www.thedacs.com/databases/roi/– “Demonstrating the Impact and Benefits of CMMI: An Update

and Preliminary Results,” Software Engineering Institute, CMU/SEI-2003-SR-009, Oct 2003


21

Typical CMMI Benefits Cited in Literature

• Reduced costs– 33% decrease in the average

cost to fix a defect (Boeing)– 20% reduction in unit

software costs (Lockheed Martin)

• Faster Schedules– 50% reduction in release

turnaround time (Boeing)– 60% reduction in re-work

following test (Boeing)

• Greater Productivity– 25-30% increase in

productivity within 3 years (Lockheed Martin, Harris, Siemens)

• Higher Quality– 50% reduction of software

defects (Lockheed Martin)

• Customer Satisfaction– 55% increase in award fees

(Lockheed Martin)


22

Agenda





23

What is Lean Six Sigma (LSS)?

• Lean Six Sigma is a powerful approach to improving the work we do

• LSS improvement projects are performed by teams

• Teams use a set of tools and techniques to understand problems and find solutions

• Lean Six Sigma integrates tools and techniques from two proven process improvement methods

+


24

What is Six Sigma?

• Six Sigma is a management philosophy based on meeting business objectives by reducing variation– A disciplined, data-driven methodology for decision making

and process improvement

• To increase process performance, you have to decrease variation

Defects Defects

Too early Too late

Delivery Time

Reduce variation

Delivery Time

Too early Too late

Spread of variation too wide compared to

specifications

Spread of variation narrow compared to

specifications

•Greater predictability in the process

•Less waste and rework, which lowers costs

•Products and services that perform better and last longer

•Happier customers


25

A General Purpose Problem-Solving Methodology: DMAIC

Define

Problem or goal statement (Y)

ControlAnalyze ImproveMeasure

• An improvement journey to achieve goals and resolve problems by discovering and understanding relationships between process inputs and outputs, such asY = f(defect profile, yield)

= f(review rate, method, complexity……)

• Refine problem & goal statements.

• Define project scope & boundaries.


26

DMAIC Roadmap

Define ControlAnalyze ImproveMeasure

Define project scope

Establish formal project

Identify needed data

Obtain data set

Evaluate data quality

Summarize& baseline data

Explore data

Characterize process & problem

Identify possible solutions

Implement (pilot as needed)

Define control method

Implement

Update improvement project scope & scale

Document

Select solution

Evaluate

Phase Exit Review

[Hallowell-Siviy 05]


27

Benchmark

Contract/Charter

Kano Model

Voice of the Customer

Voice of the Business

Quality Function Deployment

GQIM and Indicator Templates

Data Collection Methods

Measurement System Evaluation

Statistical Controls:

Control Charts

Time Series methods

Non-Statistical Controls:

Procedural adherence

Performance Mgmt

Preventive measures

DMAIC Toolkit

ControlImproveMeasureDefine Analyze

Design of Experiments

Modeling

ANOVA

Tolerancing

Robust Design

Systems Thinking

Decision & Risk Analysis

PSM Perform Analysis Model

Cause & Effect Diagrams/ Matrix

Failure Modes & Effects Analysis

Statistical Inference

Reliability Analysis

Root Cause Analysis, including 5 Whys

Hypothesis Test


28

Design for Six Sigma (e.g., DMADV)

Define VerifyAnalyze DesignMeasure

Define project scope

Establish formal project

Identify customers

Research VOC

Benchmark

Quantify CTQs

Explore data

Design solution

Develop detailed design

Develop pilot

Evaluate pilot

Scale-up design

Predict performance

Document

Refine predicted performance


29

What is Lean?

• Series of tools and techniques refined by Toyota and called the “Toyota Production System”– Called “Lean” by Womack, Jones and Roos in The Machine

That Changed the World

• Focused on increasing efficiency by eliminating non-value added process steps and wasteful practices

• Being adopted world-wide by both manufacturing and transactional based organizations

• Utilizes tools like “Value Stream Mapping,” “Just in Time” and “Kaizen”

LEAN FOCUS: ELIMINATE WASTE AND REDUCE CYCLE TIME


30

Wastes in Production

•

Typesof

Waste

CORRECTION

WAITING

PROCESSING

MOTION

INVENTORYCONVEYANCE

OVERPRODUCTION

Repair orRework Any wasted motion

to pick up parts or stack parts. Also wasted walking

Wasted effort to transportmaterials, parts, or finished goods into or out of storage, or between processes.

Producing morethan is needed before it is needed

Maintaining excessinventory of raw mat’ls,parts in process, orfinished goods.

Doing more work thanis necessary

Any non-work timewaiting for tools, supplies, parts, etc..


31

Returns

Supplier Output CustomerProcessInput

The Hidden Process

Inspectand/or Sign

Reprocess

Notsure

Re-route

OK

NoScrap

Secondlook

Return or Reprocess

Reprocess

ReturnTo

process

The Hidden Process


32

Organizational Adoption:Roles & Responsibilities

• Champions – Facilitate the leadership, implementation, and deployment

• Sponsors – Provide resources

• Process Owners – Responsible for the processes being improved

• Master Black Belts – Serve as mentors for Black Belts

• Black Belts – Lead major Six Sigma projects– Typically requires 4 weeks of training

• Green Belts – Lead minor Six Sigma teams, or serve on improvement teams under a Black Belt– Typically requires 2 weeks of training


33

A Typical Lean Six Sigma Project in Aerospace

The organization notes that systems integration has been problematic on past projects (budget/schedule overruns)

A Six Sigma team is formed to scope the problem, collect data from past projects, and determine the root cause(s)

The team’s analysis of the historical data indicates that ineffective peer reviews are leaving significant errors to be found in test

Procedures and criteria for better peer reviews are written, using best practices from past projects

A pilot project uses the new peer review procedures and criteria, and collects data to verify they solve the problem

The organization’s standard process and training is modified to incorporate the procedures and criteria, to prevent similar problems on future projects


34

Agenda





35

The Manifesto for Agile Software Development

“We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

Individuals and interactionsover processes and tools

Working software overcomprehensive documentation

Customer collaboration over contract negotiation

Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.”

- Kent Beck et al


36

Definition of Agile

• Agile is an iterative and incremental (evolutionary) approach to software development which is performed in a highly collaborative manner with "just enough" ceremony that produces high quality software which meets the changing needs of its stakeholders.

http://www.agilemodeling.com/essays/agileSoftwareDevelopment.htm


37

What is “Agility”?

• Effective (rapid and adaptive) response to change

• Effective communication among all stakeholders

• Drawing the customer onto the team

• Organizing a team so that it is in control of the work performed

Yielding …

• Rapid, incremental delivery of software


38

An Agile Process

• Is driven by customer descriptions of what is required (scenarios)

• Recognizes that plans are short-lived

• Develops software iteratively with a heavy emphasis on construction activities

• Delivers multiple ‘software increments’

• Adapts as changes occur


39

Agile is Supported by Many Methodologies

Extreme Programming (XP)

• Based on values of simplicity, communication, feedback, courage, and respect

• Start with simple solution, add complexity through refactoring

• Frequent feedback through unit, integration, and acceptance testing

• 4 dev. Phases: coding, testing, listening, designing

Agile Unified Process

• Simplified version of RUP – reduced number of disciplines

• Consists of 4 RUP phases (Inception, Elaboration, Construction, Transition)

Crystal• Frequent delivery• Reflective

improvement• Close communication

w/personal safety• Access to expert

users• Automated testing• Frequent integration• Configuration

management

SCRUM• Small teams of 6-8

people• “Backlog” defined

requirements that will be addressed in each Sprint

• Daily 15 min.Scrum meeting to discuss work for the day

• Divide projects into 30 day “Sprints”

• Review conducted at end of each Sprint to review progress and revise backlog

Adaptive • Repeating Speculate,

Collaborate, Learn cycles

• Provides for continuous learning and adaptation to changing project state

Feature Driven Dev.

• More value on design then the “code is the design”

• Model-driven• Develop feature list• Plan, Design, Build

by Feature

Dynamic Systems Development Method (DSDM)

• 3 primary phases: Pre-Project, Project Life-Cuc;e , Post-Project

• Project Life-Cycle consists of Feasibility Study, Business Study, Functional Model Iteration, Design/Build Iteration, and Implementation


40

Agenda





41

Northrop Grumman Approach:

Mission Success Requires Multiple Approaches

Process Effectiveness

Program Effectiveness

MissionAssurance & Enterprise Excellence

Operations Effectiveness

Dashboards for Enterprise-Wide Measurement

Communications & Best-Practice Sharing

Robust Governance Model (Policies,

Processes, Procedures)

Risk Management

Systems Engineering

Independent Reviews & Cost Estimates

Training, Tools, & Templates

CMMI Level 5 for Software, Systems, and

Services

ISO 9001 and AS-9100 Certification

Six Sigma


42


Institutionalizing Our Improvements

InternalBest

Practices

InternalBest

Practices

ISO/AS9100 Findings

ISO/AS9100 Findings

CMMI Appraisal Findings

CMMI Appraisal Findings

PolicyPolicy

ProceduresProcedures

ProcessProcess

eToolkiteToolkit PALPAL

WorkbenchWorkbenchStartIt!StartIt! My MS PortalMy MS Portal

Tools

Information

Checklists and Guides

Checklists and Guides

Templates and Examples

Templates and Examples

Disposition

Independent Audits

Independent Audits

•Systems/ Software Engineering Process Group

•QMS Working Group

•Program Management Advisory BoardLessons

Learned & Metrics

Lessons Learned &

Metrics

Analysis

Customer CommentsCustomer Comments

Configuration Control Board

ExternalBest

Practices

ExternalBest

Practices

Industry StandardsIndustry

Standards

Six Sigma Projects

Six Sigma Projects

We systematically analyze quality and process data and trends to determine how to improve our

processes

We systematically analyze quality and process data and trends to determine how to improve our

processes

We improve our process assets based on internal and external

best practices

We improve our process assets based on internal and external

best practices Deploye

d to program

s

Increasing program efficiency

msCASmsCAS

PCDBPCDB


43


Lessons Learned

Based on over 20 Northrop Grumman CMMI Level 5 and Lean Six Sigma organizations

• Multiple improvement initiatives helps encourage a change in behavior as opposed to “achieving a level”– Reinforces that change (improvement) is a way of life

• The real ROI comes in institutionalizing local improvements across the wider organization– CMMI establishes the needed mechanisms

• CMMI and Lean Six Sigma compliment each other– CMMI can yield behaviors without benefits– Lean Six Sigma improvements based solely on data may miss innovative

improvements (assumes a local optimum)

• Training over half the staff as Lean Six Sigma Green Belts has resulted in a change of language and culture– Voice of Customer, data-driven decisions, causal analysis, etc.– Better to understand/use tools in everyday work than to adopt the

“religion”

Documents

Process Improvement Workshop Process Improvement in the Aerospace Industry Caltech Aerospace Seminar 13 April 2009 Rick Hefner, Ph.D. Director, Process