DMAIC Version 3, Jan., 2005 Page 1 Caterpillar Confidential: Green Measure Module 2.1 Determine What to Measure

DMAIC Version 3, Jan., 2005

Page 1

Caterpillar Confidential: Green

Measure

Module 2.1

Determine What to Measure

Green Belt – Define & Measure - Page 2DMAIC Version 3


Suppliers Process Inputs Business Processes Process OutputsCritical

CustomerRequirements

Important decisions based on linking

customer expectations to process

performance

Input Measures

Process Measures

Output Performance Measures

Customer Value

Performance Measures – Customer Value Achieved?



Input, Process, and Output Indicators

Critical Customer RequirementsCritical to the Process

InputIndicators

ProcessIndicators

OutputPerformance

Indicators

Cost per transaction Time per activity Amount of rework Turnaround time Variability of an activity

Percent defective Number of errors Total response time Invoice/billing accuracy Revenue

These should all appear on your process map

# of customer inquiries Type of customer inquiries # of orders # of positions open Type of position open Accuracy of the credit analysis Timeliness of the contract

submitted for review


Page 5


Measure

Module 2.3

Manage Measurement



Data Collection

Measurement management starts with a data collection methodology.

IdentifyMeasures

Step 1Develop operational

definitions for measure

Step 2Develop measurement plan

Step 3Collect data

Step 4Display and Evaluate data



Two Basic Types of DataBefore data collections starts, classify the data into different types: continuous or discrete. This is important because it will:

– Provide a choice of data display and analysis tools

– Dictate sample size calculation

– Provide performance or cause information

– Determine the appropriate control chart to use

– Determine the appropriate method for calculation of Sigma

Continuous or variable

Measured on a continuum

Objective• Time• Money• Weight• Length

Subjective• Satisfaction• Agreement• Extent• Type of error

Discrete, categorical, or attribute

Count or categories

Objective• Count defects• # approved• # of errors• Type of document

Subjective• Yes/No• Categories• Service

performance rating (good, poor)

• Satisfaction• Agreement



Step 1: Develop Operational Definitions

Step 1: Operational DefinitionAn operational definition is a concept that helps guide the team’s thinking on what they need to measure as well as the key attributes of the measure: what, how, and who. It provides the foundation for the team to reach agreement and build consistency and reliability into data collection. This helps ensure any person using the agreed-on definition will be measuring the same thing.

– Provides everybody with the same meaning.

– Ensures that consistency and reliability are built in up front.

– Describes the scope of the measure (what is included and what is not included).



Step 2: Develop a Measurement Plan

Example: Cycle time for loan application processing

Considerations for other data that should be collected at the same time:How will you display the data? What do you want to do with the data after it is collected?How do you want to stratify the data?What data might you need to identify and verify root cause?

Data collection is a balance between time, money and accuracy (getting the data you need).

Performance Measure

Operational Definition

Data Source and Location

Sample Size

Who Will Collect the

Data

When Will Data be

Collected

How Will Data be

Collected

Other Data that should

be Collected at the same

time Time to

process a loan

application

Fax date, time

Decision fax date, time

Loan applications

Representative fax center

289 Tim Smith Dave Mann

During the first week of the month, 10/1/99 to

10/7/99

Randomly selected

from September

‘99

Type of loan Amount of

loan Dealer

Time of day Day of week

Each 6 Sigma improvement team should complete a measurement plan that contains the following information:



Step 2: Develop a Measurement Plan

Example: Industrial Pre-Treat Cost Reduction

IPT Cost Reduction - Measurement Plan

ppm dosageppm of IC -1187 per treatment

Tony's logsheets since Nov Ericweek of 5/15/06

compiled from log book

volume processedgallons discharged per treatment

Tony's logsheets since Nov Ericweek of 5/15/07

compiled from log book

IPT CostsBase chargesvariable charges

C ity of Thomasville's invoices

36+ mos Ericweek of 5/15/08

compiled from invoices

Machine Hoursmachine hours produced per month

Mfg-Pro 36+ mos Heatherweek of 5/15/09

compiled from MFG-Pro

Used Oil Volume

Used oil sent for recycling by CatUsed oil sent for recycling by IPT

Atlantic Industrial logs 18 mos Ericweek of 5/15/10

compiled from invoices

significant eventsECM Salt dumps, Central Coolant System cleanouts

Tony's logsheets Tony as needed

Sample SizeData Source & LocationWho will

collect the data?

When will data be

collected?Performance Measure Operational Definition

How will data be collected?



Compressed Air Efficiency - Measurement Plan

Energy Usage Monthly kWH UsageC ity of Thomasville Electricity Invoice

24 months Eric NeihausJ an '04 - Dec

'05collect invoices

Energy CostElectricity Charges x Tax Rate

C ity of Thomasville Electricity invoice 24 months Eric Neihaus

J an '04 - Dec '05

collect invoices

Machine HoursTotal Monthly Machine Hours for Thomasville facility

MFG-Pro 25 monthsHeather Logan

J an '04 - Dec '05

MFG-Pro reports

Energy Usage / Mach Hr= Monthly kWH usage / Monthly Machine Hours

from above 26 monthsHeather Logan

J an '04 - Dec '05

Mini-tab

Energy Consumption per CompressorkWH Energy Meter 7 d

Air Flow Demand psig Air Flow Meter 7 d

Peak Demands - Historical City of Thomasville 12 m

Peak Demands - Calculated Energy/flow meters 7 d

Performance Measure Operational DefinitionHow will data be

collected?Sample

SizeData Source & Location

Who will collect the

data?

When will data be

collected?




Page 13


Measure

Module 2.6

Gage R&R



How might measurementvariation affect these decisions?

Verify product/process

conformity to specifications

Verify product/process

conformity to specifications

Assist incontinuous

improvement activities

Assist incontinuous

improvement activities

What if the amount of measurement variation

is unknown

?

Process

Measurement

Process

Measurement

Measurement variation can make our process capabilities appear worse than they are

Why Worry about Measurement Variation?

Consider the reasons why we measure:



The Importance of Measurement System Analysis (MSA)

Is the importance of conducting an MSA being overlooked?– Many resources can be wasted trying to improve a process

when a major source of variability is the measurement system

– More consideration needs to be given to the measurement system before beginning process improvement work

The measurement system is only one source of variability when measuring a product or process

The purpose of a measurement system is to better understand the sources of variation that can influence the results produced by the process under investigation



Sources of Variation

Product Variability(Actual variability)

MeasurementVariability

Total Variability(Observed variability)

Actual Product/Process Variation

Total Observed Variation

Measurement Variation

Common cause variation

Special causeVariation

Within partvariation

Variationdue to Operator

Variationdue to Gage



T ru e V a lu es

In s trum e nt A ccu ra cy B ia s

R e p e ata b ility R e p ro d u c ib ility

G a g e R & R S ta b ility L in e a rity D isc rim in a tion

M e a sure m e nt E rro r

O b serve d M e asu re m e n ts

Gage R&R is the percentage of the total process variation related to the measurement system for repeatability and reproducibility

Gage R&R is the percentage of the total process variation related to the measurement system for repeatability and reproducibility

Generally recognized criteria for gage acceptability is when Gage R& R variability yields:

Under 10%: Acceptable gage

10% to 30%: May be acceptable

Over 30%: Gage is unacceptable and should be corrected or replaced

Generally recognized criteria for gage acceptability is when Gage R& R variability yields:

Under 10%: Acceptable gage

10% to 30%: May be acceptable

Over 30%: Gage is unacceptable and should be corrected or replaced

Components of Measurement Error

The sources of variability for the measurement system are:



MSA for Coin Sac Depth Gage 0320-000336

TL = 14.56

Gage R&R

%ContributionSource VarComp (of VarComp)

Total Gage R&R 2.00E-06 3.38 Repeatability 1.35E-06 2.29 Reproducibility 6.48E-07 1.10 Operator 6.48E-07 1.10

Part-To-Part 5.70E-05 96.62 Total Variation 5.90E-05 100.00

StdDev Study Var %Study Var %ToleranceSource (SD) (5.15*SD) (%SV) (SV/Toler)

Total Gage R&R 1.41E-03 7.28E-03 18.40 14.56 Repeatability 1.16E-03 5.98E-03 15.12 11.97 Reproducibility 8.05E-04 4.15E-03 10.48 8.29 Operator 8.05E-04 4.15E-03 10.48 8.29

Part-To-Part 7.55E-03 3.89E-02 98.29 77.79 Total Variation 7.68E-03 3.96E-02 100.00 79.14





Step 3: Collect Data

Follow the plan — note any deviations from the plan

Consistency — avoid bias

Observe data collection

Does anyone have any experience of data collection?



Discrete Data - Example



SamplingUsing a sample of data you draw conclusions about the entire population of data. This is known as “statistical inference.” Sampling saves costs and time. Sampling provides a good alternative to collecting all the data. Identifying a specific confidence level allows us to make reasonable business decisions.

Parameters:

Sampling From a Population

EntirePopulation

of Data Sample

StatisticalInference

Statistics:X, S, etc.

Analysis



Different situations which dictate sampling techniques:

• To analyze and control a process

• To describe a large population (i.e., types of customers and buying behavior)

Systematic Process Sampling

Typical DescriptiveStatistics:

Random Samplingfrom a Population

SystematicProcess Sampling

X X XSample

X X X XSample

Average cycle time (xbar)No. of defects

proportion defectivestandard deviation (s)

X X X

X

XX

X



Sampling Considerations Where

– Location in the process where process steps directly affect outputs (strong relationship)

– Maximize opportunity for problem identification (cause data)

Frequency– Dependent on volume of transactions and/or activity

– Unstable process—more frequently (use systematic or subgroup sampling)

– Stable process—less frequently (use sample size formula)

– Dependent on how precise the measurement must be to make a meaningful business decision

Considerations– Is the sample representative of the process or population?

– Is the process stable?

– Is the sample random?

– Is there an equal probability of selecting any data point?

– The answer to each of these questions must be yes before we can draw statistically valid conclusions.





Step 4: Display & Evaluate Data

Has your data collection method:– given you dependable data?– provided consistent information throughout the data

collection period?– provided a reliable set of data?– given representative data?

If you repeat the data collection will you get similar results?

Does the data collected provide the information you need?


Page 34


Measure

Module 2.4

Control Charts



Variation

Variation means that a process does not produce exactly the same result every time the product or service is delivered

Variation exists in all processes Measuring and understanding variation in our business processes helps

identify specifically what the current level of performance is and what needs to change in order to reduce the variability and therefore reduce the defects delivered to customers

Data Variation



Common Cause Variation

There are always inherent chance causes responsible for natural variation in all processes due to “normal” variation in materials, environments, methods, etc. (common cause)

Variation within a stable pattern of chance causes is inevitable



Special Cause Variation

Once we have an indication of a shift outside a stable pattern of variation, we must discover the reason for the shift (special cause)

We want to remove the influence of a special cause if it is adversely affecting product-process quality

If the special cause influence is improving product-process quality (e.g., 6 Sigma projects), we want to permanently capture its effect



What Causes Variation?

Suppliers Process Inputs Business Process Process OutputsCritical

CustomerRequirements

Variation in the output of

processes causes defects

DefectsRoot cause analysis of

variation leads to permanent defect

reduction



Charting Variation – Control Charts

14151617181920212223242526

M T W Th F M T W Th F M T W Th F M T W Th F M

Co

mm

uti

ng

Tim

e (

min

s)

Days

UCL

CL

LCL



Control Charts - Basics

Control charts

Have the ability to indicate the presence of special causes that upset our processes

Help us to detect, diagnose, and correct process problems in a timely fashion

Provide an easy to understand visual indicator of process performance



Reacting to Data/Tweaking

Control Charts

Recognize the extent of variation that now exists so that we do not overreact to common cause variation (tweaking)

Study the process to identify sources of variation and then act to eliminate or reduce those sources of variation or to mitigate their effects

Special causes can be identified with an effective diagnostic program

Common causes could not be removed or drastically reduced without making fundamental process changes



Reacting to Data/Tweaking

Tweaking Tweaking

Normal

ReactingTo Data


Page 45


Measure

Module 2.7

Process Capability



Process Capability – What is it?

Most measures have some target value and acceptable limits of variation around the target

The extent to which the “expected” values fall within these limits determines how capable the process is of meeting its requirements

Consider key measures of process performance in:– Help desk responsiveness– Laminate flatness– Pump flow rate

Quantifiable comparison of Voice of the Customer(spec limits) to Voice of the Process (control limits)Quantifiable comparison of Voice of the Customer

(spec limits) to Voice of the Process (control limits)



Process Capability Ratio – Cp

Ratio of total variation allowed by the specification to the total variation actually measured from the process

Typical goals for Cp are greater than 1.33 (or 1.67 for safety items)

If Cp < 1, then the variability of the processis greater than the specification limits

If Cp < 1, then the variability of the processis greater than the specification limits



Process Capability Ratio – Cpk

This index accounts for the dynamic mean shift in the process – the amount that the process is off target

σ̂σ̂ 3

LSLxor

3

xUSLMinCpk



Uses of Capability Analysis

Performed on new equipment as part of the qualification and approval process

Performed on existing processes as a means of establishing a baseline of current operations (so it’s possible to tell when improvement has occurred)

When done periodically, is a means of monitoring wear and tear on equipment, and deterioration of a process for whatever reason (material, personnel, environment, etc.)

Can be done on any process that has a spec. established, manufacturing or transactional (spec. is needed for the values in numerator), and has a capable measuring system (needed for valid values in denominator)



Preparing for Capability Analysis

When beginning to measure/monitor a parameter always: Calibrate measuring system (unless done recently)

– Use a standard traceable to NIST (National Institute of Standards and Technology)

– One trained technician Perform Measuring System Analysis (MSA)

– Group of actual parts (use parts from the high and low ends of the tolerance range, even rejected parts)

– Group of actual machine operators– Multiple ‘blind’ and random runs

The process should be statistically in-control



Discrete Data – ExampleSigma Level



Cos

t

Sigma

Optimum

Impact of Quality Level on Cost

Costs Are Always Kept in Mind

There is an optimum quality level beyond which the costs of quality improvement exceed the expected cost savings from a reduced number of defects



Rolled Throughput Yield vs. First Pass Yield

Using Final Test “First Pass Yield” ignores the hidden factory. Final test performance is a function of inspection/test or prevention.

Using Final Test “First Pass Yield” ignores the hidden factory. Final test performance is a function of inspection/test or prevention.

66% is not 90% ... why not?

Process1 2 3

Rolled Yield 81 % 73 %

4

66 %

Final Test

=90%Yield

90%Yield

90%Yield

90%Yield

90%Yield

(Yield After Inspection or Test)

Scrap

InputsInputs

Hidden FactoryHidden Factory

NOT

OK

OperationOperation First Pass Yield

First Pass Yield

OK 90% Customer QualityInspectInspect

Rework



Rolled Throughput Yield Example – Statement

Airtronics has a seven-step process for taking a purchase requisition and turning it into an issued Purchase Order

The seven steps have the following first pass yield rates:– Step 1: 0.981– Step 2: 0.972– Step 3: 0.943– Step 4: 0.892– Step 5: 0.943– Step 6: 0.904– Step 7: 0.963

What is the Rolled Throughput Yield for this 7 Step Process?



Rolled Throughput Yield Example – Solution

The Rolled Throughput Yield for the seven step process is:

Step 1 x Step 2 x Step 3 x Step 4 x Step 5 x Step 6 x Step 7 = Rolled Throughput Yield

0.981 x 0.972 x 0.943 x 0.892 x 0.943 x 0.904 x 0.963 = 0.658

Conclusion: 66 percent of the purchase requisitions make it through the process with no errors the first time



Why 6 Sigma is the Goal

Rolled Throughput Yield (RTY) is the probability that a product will pass through the entire process without rework and without any defects. If you have a complex product and/or process, 6 sigma yield will be needed at each process step or for each part in order to have RTY of greater than 90 percent.

Impact of Complexity on Rolled Throughput Yield

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0 1,000 2,000 3,000 4,000 5,000 6,000 7,000 8,000 9,000 10,000

Number of Parts/Process Steps

Ro

lled

Th

rou

gh

pu

t Y

ield

Impact of Complexity on Rolled Throughput Yield

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0 1,000 2,000 3,000 4,000 5,000 6,000 7,000 8,000 9,000 10,000

Number of Parts/Process Steps

Ro

lled

Th

rou

gh

pu

t Y

ield

4

5

6


Page 58


Measure

Module 2.5

Week 1 Financials**Note: This module is for Caterpillar internal use, so this may not be applicable to Caterpillar Suppliers and Dealers. Please check with your company Deployment Champion regarding this subject.



CommittedLeadershipCommittedLeadership

QuantifiableMeasures& Results

Incentives &

Accountability

Full Time6 Sigma

Team Leaders

StrategyIntegration

Customer &

MarketNetwork

BusinessProcess

Framework

CommittedLeadershipCommittedLeadership

QuantifiableQuantifiableMeasuresMeasures& Results& Results

Incentives &

Accountability

Full Time6 Sigma

Team Leaders

StrategyIntegration

Customer &

MarketNetwork

ProcessFramework

CSF: Quantifiable CSF: Quantifiable Measures & Measures & ResultsResults:

6 Sigma projects should be designed to pay their way…

Quantifiable benefits must be traceable to the enterprise bottom line

Business

Why We Measure - Recipe for Success



Why We Measure the Financial Impact

Primary reason is to ensure financial benefits from projects are real…validates success and establishes credibility

What gets measured, gets done Drives bottom-line focus Forces value-add mind set Facilitates filtering and prioritization of projects



Establishing the Financial Baseline

Baselines reflect the current “as-is” state of the process– Current revenue or cost structure– Use budgetary approach– Not Business Plan or Prior Year Base Period

Determine the ENTERPRISE resources consumed at each step of the process– Head count, material, overtime, external charges

(contracts, etc.), expediting charges, etc.



Establishing the Financial Baseline

Baselines should concentrate on the relevant costs – those costs that will be impacted by your project

For example, a project targeted at reducing packaging costs could ignore freight, forwarding, and duty costs if they were unaffected by the packaging

It is still important to understand all costs to be sure the project is targeted in the most appropriate area


Page 63


Measure

Module 2.8

Stakeholder Management/Communication Plan



What is Stakeholder Management?

Identify Stakeholders

Understand Stakeholders

Influence Stakeholders



Keep stakeholders aware and involved in your project…

Gate Reviews

Update Meetings

Gallery Walks

Communication is the Key to Overcoming Resistance



Critical Success Factors

Enlist visible senior management sponsorship/support

Budget plenty of time up-front for stakeholder identification and communication

Get the right people involved Provide consistent and on-going communications Communicate continuously and honestly Identify issues early to resolve them


Page 68


Measure

Module 2.10

Business Risk Management Checkpoint



Define

Measure

Analyze

Improve

Control

• BRM issues included in problem statement• All customers identified and included (e.g.

regulators, shareholders, etc.) stakeholder analysis

• ALL CCR’s appropriately prioritized

• Emphasize importance of minimizing variation

• Use BRM checkpoint process

• Broaden view of customer - prevent team from automatically eliminating activity not directly related to paying customers


• Use risk management tools to help balance risk and control

• Assess impact of proposed change on all stakeholders


Charter Team

Determine CCR’s

Map Process

Identify Possible Causes

Narrow to Root Causes

Quantify Opportunity

Identify and Test Solution

Refine Solution

Close Project

Pilot Solution

Implement Solution

Collect Data

Define Defect

Describe Variation

Prepare Process Control Plan • Use BRM checkpoint process

• Emphasize importance of ongoing monitoring

BRM



BRM Checkpoints – Measure

For Critical Customer Requirements (CCRs) that have the greatest impact, have defects and related measures been defined appropriately?

When analyzing process performance, are the risk factors aligned with your organization’s risk management strategies?

How has the risk impact changed (i.e., has the updated Process Risk Impact Worksheet indicated a change in the risk profile)?

Documents

DMAIC Version 3, Jan., 2005 Page 1 Caterpillar Confidential: Green Measure Module 2.1 Determine What to Measure