MSA - Measurement Systems Analysis

Six Sigma Champion Training 1

Measurement Systems Analysis

• Champion Training

Improve

Analyze

Measure

Control


Learning Objectives

• Understand the language of Measurement

• Show the importance of Measurement

• Walk away knowing how to perform a Gage R&R and how to interpret results

• Share some lessons learned


• They are subject to Variation.

• What could be the source of this variation?

• Why do Measurements Vary?

CLASS EXERCISE : Break into teams. Do a cause-effect Diagram to determine the causes of variation in Measurement.

Since Measurement systems represent a sub-process within a process...


Sources of Measurement Variation

Measurement System Error

Temp Fluxctuation

Line Voltage Variation

Vibration

Cleanliness

Humidity

Algorithm Instability

Electrical Instability

Wear

Mechanical Integrety

Operator Technique

Standard Procedure

Suff icient Work time

Maintenance Standard

Calibration Frequency

Operator Training

Ease of use

Density

Conductivity

Hardness

Corrosion

Weight

Dimension

Temperature

Cleanliness

Wear

Stability

Resolution

Calibration

Precision

Design

Temperature

Cleanliness

VisionDexterity

Know ledgeCoordination

SpeedInterpretation

Calibration ErrorAttention

FatigueProcedure

Men

Machines

Materials

Methods

Measurements

Environment

Measurement System C&E Matrix


Possible Sources of Process Variation

Long-term

Process Variation

Short-term

Process Variation

Variation

w/i sample

Actual Process Variation

Stability LinearityRepeatability Accuracy

Variation due

to gage

Variation due

to operators

Measurement Variation

Observed Process Variation

We will look at “repeatability” and “reproducibility” as these are the primary contributors to measurement error.We will look at “repeatability” and “reproducibility” as these are the primary contributors to measurement error.

Reproducibility


Knowledge to be obtained

• How big is the measurement error?

• What are the sources of measurement error?

• Is the gage stable over time?

• Is the gage capable for this process?

• How do we improve the measurement system?


Sources of Variation

Product Variability

(Actual variability)

Product Variability

(Actual variability)

MeasurementVariability

MeasurementVariability

Total Variability(Observed variability)

Total Variability(Observed variability)


Effects of Measurement Error

AveragesAverages

VariabilityVariability

total product measurement

total product measurement2 2 2

Measurement System Bias —

Determined through “Accuracy Study”

Measurement System Variability —

Determined through “R&R Study”


If you want to decrease your gage error take advantage of the standard error square root of the sample:

Example: Gage error of 50% can be cut in half if your point estimate is a sample of 4 data points.

Example: Gage error of 50% can be cut in half if your point estimate is a sample of 4 data points.

THIS IS USED AS A SHORT TERM APPROACH TO PERFORM A STUDY, BUT YOU MUST FIX THE GAGE.

n= sample size

xx

n

Work Around Gage Error


Terminology

• Location related terms:– True value– Bias– Linearity

• Stability (over time)• Variation related terms

– Repeatability– Reproducibility– Linearity


• True value:– Theoretically correct value – unknown and unknowable– Reference standards– NIST standards

• Bias– Distance between average value of all measurements

and true value– Amount gage is consistently off target– Systematic error or offset


BIAS — Is the difference between the observed average of the measurement and the reference value. The reference-value is the value that serves as an agreed-upon reference. The reference value can be determined by averaging several measurements with a higher level (e.g., metrology lab) of measuring equipment.

Warning: Don’t assume your metrology reference is gospel. Observed

Average Value

ReferenceValue

ACCURACY IS THE SAME AS BIAS

BIAS Definition


LinearityDifference in the accuracy values of a gage through the expected operating range of the gage

5040302010

55

45

35

25

15

5

Standard

Tria

ls

R-Squared = 0.981

Y = 0.934227 + 0.994959X

Regression Plot

5040302010

55

45

35

25

15

5

Standard

Tria

ls

R-Squared = 0.982

Y = 0.245295 + 0.99505X

Linearity is Not Good

Good Linearity Bad Linearity


Stability

• The distribution of measurements remains constant and predictable over time for both mean and standard deviation

• Total variation in the measurements obtained with a gage, on the same master or master parts, when measuring a single characteristic over an extended time period.

• Evaluated using a trend chart or multiple measurement analysis studies over time


Stability (drift) Definition

Stability — Is the total variation in the measurement obtained with a measurement system (test / gage ) on the same master parts when measuring a single characteristic over an extended time period.

Time-1

Time-2

time

Magnitude

StabilityPoints to the frequency of Mean center CalibrationPoints to the frequency of Mean center Calibration


MS G O2 2 2

• Total variation in the measurement system• Measure of natural variation of repeated

measurements• Terms: Random Error, Spread, Test/Retest

error• Repeatability and Reproducibility


G

G

R

d

2*

Repeatability

• The inherent variability of the measurement system• Variation in measurements obtained with a gage when

used several times by one operator while measuring a characteristic on one part.

• Estimated by the pooled standard deviation of the distribution of repeated measurements

• Repeatability is less than the total variation of the measurement system


Repeatability Definition

REPEATABILITY

Repeatability — The variation in measurementsobtained with one measurement instrument when used several times by one appraiser whilemeasuring the identical characteristic onsame part.


O

O

R

d

2*

Reproducibility

• Operator variability of the measurement system• Variation in the average of the measurements

made by different operators using the same gage when measuring a characteristic on one part

• Must be adjusted for gage variation• Reproducibility is less than the total variation of

the measurement system


Reproducibility — Is the variation in the average of the measurements made by different appraisers using the same measuring instrument when measuring the identical characteristic on the same part.

Reproducibility

Operator-A

Operator-C

Operator-B

Reproducibility Definition


The Nature of Process Variation

54321

4321

Precise but not AccuratePrecise but not Accurate

Accurate but not PreciseAccurate but not Precise

. . . . . .Test equipment MUST be a least 10 times more accurate & precise then what’s being tested

Rule of thumb:


Measurement System DiscriminationLeast count should be at most one-tenth of the total

process capability or tolerance (6 sigma)

– Process capability 10 Max Least count 1Part to Part variation must be greater than the smallest

unit of measureRange control chart provides best indication of

inadequate discrimination

– Occurs when only 1,2, or 3 possible values for the range within the control limits exists

Number of Distinct Categories equals part sigma/ total

gage sigma 1.41.

Least count should be at most one-tenth of the total process capability or tolerance (6 sigma)

– Process capability 10 Max Least count 1Part to Part variation must be greater than the smallest

unit of measureRange control chart provides best indication of

inadequate discrimination

– Occurs when only 1,2, or 3 possible values for the range within the control limits exists

Number of Distinct Categories equals part sigma/ total

gage sigma 1.41.


Measurement System Capability

Gage Capability

Addresses what percent of the tolerance or process capability is taken up by measurement error.

Best case: 10% Acceptable: 30% Includes both repeatability and reproducibility

– Operator Unit Trial Experiment

GR RTolerance

MS&. *

515 Usually expressed

as percent

Usually expressed as percent


CUSL LSL

pAct

Act

6 Act Obs MS 2 2

CUSL LSL

pAct

Obs MS

6 2 2

where

CpMeasurement Error

Effect on Capability Index

• We know that

• Therefore:


R&R Effect on Capability

0.0

1.0

2.0

3.0

4.0

5.0

6.0

0.5 0.6 0.7 0.8 0.9 1.0 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 2.0

Observed Cp

Ac

tua

l Cp 0%

10%

20%

30%

40%50%

60%

70%

%R&R

70% 60% 50%

40%

30%

10%


Types of R&R Studies

• Variable Gage R&R– Numbers– Units of measure

• Attribute Gage R&R– Subjective (cosmetic defects)– Scatter of defects– feel/visual


The Inspection Exercise

The Necessity of Training Farm Hands for First

Class Farms in the Fatherly Handling of Farm Live

Stock is Foremost in the Eyes of Farm Owners.

Since the Forefathers of the Farm Owners Trained

the Farm Hands for First Class Farms in the

Fatherly Handling of Farm Live Stock, the Farm

Owners Feel they should carry on with the Family

Tradition of Training Farm Hands of First Class

Farmers in the Fatherly Handling of Farm Live

Stock Because they Believe it is the Basis of Good

Fundamental Farm Management.

Task: Count the number of times the 6th letter of the alphabet appears in the following text.


Try To Always Convert Attribute To VariablesTry To Always Convert Attribute To Variables

Examples:

End Disk Height

Likert Scale

Leak Rate (go/no go)

Mass Spec

Convert Data


Basic Terms

EV= Equipment Variation (Repeatability)

AV= Appraiser Variation (Reproducibility)

R&R= Repeatability & Reproducibility

PV= Part Variation

TV= Total Variation of R&R and PV

K1-Trial, K2-Operator, & K3-Part Constants

EV= Equipment Variation (Repeatability)

AV= Appraiser Variation (Reproducibility)

R&R= Repeatability & Reproducibility

PV= Part Variation

TV= Total Variation of R&R and PV

K1-Trial, K2-Operator, & K3-Part Constants


Generally two or three operators

Generally 10 units to measure

Each unit is measured 2-3 times by each operator

Gage R&R study


Preparation for a Measurement Study

• Determine if reproducibility is an issue. If it is, select the number of operators to participate.

• Operators selected should normally use the measurement system.

• Select samples that represent the entire operating range.• Gage must have graduations that allow at least one-tenth of

the expected process variation.• Insure defined gaging procedures are followed.• Measurements should be made in random order.• Study must be observed by someone who recognizes the

importance of conducting a reliable study.


Procedure for Performing R&R Study

• Calibrate the gage, or assure that it has been calibrated.• Have the first operator measure all the samples once in random

order.• Have the second operator measure all the samples once in

random order.• Continue until all operators have measured the samples once

(this is Trial 1).• Repeat above steps for the required number of trials.• Use GR&R form to determine the statistics of the study.

– Repeatability, Reproducibility & %GR&R– Standard deviations of each of the above– % Tolerance analysis

• Analyze results and determine action, if any.


Guidelines

% R&R Results<5% No issues

10% Gage is OK

10% – 30% Maybe acceptable based upon importanceof application, and cost factor

Over 30% Gage system needs improvement/correctiveaction

% R&R Results<5% No issues

10% Gage is OK

10% – 30% Maybe acceptable based upon importanceof application, and cost factor

Over 30% Gage system needs improvement/correctiveaction

Variable Gage R&R


PT1 PT2 PT3 PT4 PT5 PT6 PT7 PT8 PT9 PT10OP/TRIAL

0.65 1.00 0.85 0.85 0.55 1.00 0.95 0.85 1.00 0.60 A1

0.60 1.00 0.80 0.95 0.45 1.00 0.95 0.80 1.00 0.70 A2

0.55 1.05 0.80 0.80 0.40 1.00 0.95 0.75 1.00 0.55 B1

0.55 0.95 0.75 0.75 0.40 1.05 0.90 0.70 0.95 0.50 B2

0.50 1.05 0.80 0.80 0.45 1.00 0.95 0.80 1.05 0.85 C1

0.55 1.00 0.80 0.80 0.50 1.05 0.95 0.80 1.05 0.80 C2

Xbar & R Minitab Example

Using Aiag49:mtw Data File

Specification: 0.6 - 1.0 mm

Process Variation: 1.6 mm

Reference QS Measurement System Analysis Manual

Gasket Thickness Study


Misc:

Tolerance:Reported by :

Date of study :Gage name:

1.11.00.90.80.70.60.50.40.3

321

Xbar Chart by Operator

Sam

ple

Mea

n

X=0.80753.0SL=0.8796

-3.0SL=0.7354

0.15

0.10

0.05

0.00

321

R Chart by Operator

Sam

ple

Ran

ge

R=0.03833

3.0SL=0.1252

-3.0SL=0.000

10 9 8 7 6 5 4 3 2 1

1.11.00.90.80.7

0.60.50.4

Gasket

OperatorOperator*Gasket Interaction

Ave

rage

123

321

1.11.00.90.80.7

0.60.50.4

Operator

Response by Operator

10 9 8 7 6 5 4 3 2 1

1.11.00.90.80.70.60.50.4

Gasket

Response by Gasket

%Total Var%Study Var%Process %Toler

Part-to-PartReprodRepeatGage R&R

200

100

0

Components of Variation

Perc

ent

Gage R&R (Xbar/R) for Thickness


Gage R&R Study for Thickness – XBar/R Method

Source Variance StdDev 5.15*Sigma

Total Gage R&R 2.08E-03 0.045650 0.235099 Repeatability 1.15E-03 0.033983 0.175015 Reproducibility 9.29E-04 0.030481 0.156975 Part-to-Part 3.08E-02 0.175577 0.904219 Total Variation 3.29E-02 0.181414 0.934282

Source %Contribution %Study Var %Tol %Process

Total Gage R&R 6.332 25.164 58.77 14.69 Repeatability 3.509 18.733 43.75 10.94 Reproducibility 2.823 16.802 39.24 9.81 Part-to-Part 93.668 96.782 226.05 56.51 Total Variation 100.000 100.000 233.57 58.39

Number of distinct categories = 5


Calculation Explanation

5.15 Sigma = 5.15 the factor standard deviation. 5.15 was developed empirically to approximate the gage population distribution variation.

% Contribution = Percent contribution of each factor based upon the variance. Repeatability = 100 repeatability variance/ total variation variance.

% Study Variation = 5.15 the factor standard deviation divided by 5.15 the total variation standard deviation. Repeatability = 100 5.15 repeatability standard deviation/ 5.15 total variation standard deviation.

% Tolerance = 5.15 the factor standard deviation divided by the tolerance. Repeatability = 100 5.15 repeatability standard deviation/tolerance.

5.15 Sigma = 5.15 the factor standard deviation. 5.15 was developed empirically to approximate the gage population distribution variation.

% Contribution = Percent contribution of each factor based upon the variance. Repeatability = 100 repeatability variance/ total variation variance.

% Study Variation = 5.15 the factor standard deviation divided by 5.15 the total variation standard deviation. Repeatability = 100 5.15 repeatability standard deviation/ 5.15 total variation standard deviation.

% Tolerance = 5.15 the factor standard deviation divided by the tolerance. Repeatability = 100 5.15 repeatability standard deviation/tolerance.


Calculation Explanation

% Process Variation = 5.15 x the factor standard deviation divided by the process variation. Repeatability = 100 x 5.15 repeatability standard deviation/ process variation.

Number of Distinct Categories = part standard deviation divided by the total gage R&R standard deviation times 1.41.

% Process Variation = 5.15 x the factor standard deviation divided by the process variation. Repeatability = 100 x 5.15 repeatability standard deviation/ process variation.

Number of Distinct Categories = part standard deviation divided by the total gage R&R standard deviation times 1.41.


Attribute Data ExampleUsing A_grr.xls file

• Metric is the % error against known population deemed good by local experts

• Attribute legend can be the defect codes• If appraiser % is less than 100% training is required, focus

on area of weakness• 100% is the target for screen effectiveness• Use this to prove measurement system capability prior to

task assignment• Select the 5 vital few (80-20 rule) to conduct GR&R


Instructions:

1) The following spreadsheet is used to calculate an Attribute GR&R Effectiveness, in which up to 30 samples can be evaluated, using 2 or 3 operators.

2) In the worksheet fill in the appropriate information in the Scoring Report section andenter the type of Attributes you are evaluating in the Attribute Legend section. YOU MUST ENTER THE INFORMATION IN THE ATTRIBUTE LEGEND SECTION OR THE SPREADSHEET WILL NOT WORK. The attributes can be either alpha or numeric, e.g. Yes, No; pass, fail; go, stop; or 1, 2. You must be consistent throughout the form and spell properly, anything will work as the spreadsheet compares what is in each cell.

3) If you or an expert has selected samples to be evaluated and you know what attributes thesesamples are, enter this information in the Attribute sample column. This will enable you to determine how well each operator can evaluate a set of samples against a known standard. You do notneed to enter information in this column for the spreadsheet to work.

4) You do not have to specify how many operators or the # of samples that you will be evaluating during the test. Simply enter the data into the spreadsheet under the specific operator. Rememberthe attributes must be spelled properly or the spreadsheet will not analyze the data correctly.

5) To print a copy of the report click on the Print Report icon.6) To delete the data in the spreadsheet, click on the Delete Data icon.7) To delete all and begin a new test, click on the Delete All icon8) To see a Demo of the Attribute GR&R Effectiveness spreadsheet, click on the Demo icon.

Move around the spread sheet to see the data. When you are finished click the Delete All icon to delete all data to begin entering your own data.

Attribute Gage R & R Effectiveness



SCORING REPORTAttribute Legend DATE:

1 pass NAME: 3/10/96

2 fail PRODUCT: Allied Employee

SBU: 3313 Spark Plug

TEST CONDITIONS: F&SP

Known Population Operator #1 Operator #2 Operator #3 Y/N Y/NSample # Attribute Try #1 Try #2 Try #1 Try #2 Try #1 Try #2 Agree Agree

123456789

101112131415161718192021222324252627282930

% APPRAISER SCORE(1) -> #DIV/0! #DIV/0! 0%

% SCORE VS. ATTRIBUTE(2) -> Known Known Known

SCREEN % EFFECTIVE SCORE(3) -> #DIV/0!

SCREEN % EFFECTIVE SCORE vs. ATTRIBUTE (4) -> #DIV/0!



SCORING REPORTAttribute Legend DATE: 3/10/96

1 pass NAME: Allied Employee

2 fail PRODUCT: 3313 Spark Plug

SBU: 3313 Spark Plug

TEST CONDITIONS: F&SP

Known Population Operator #1 Operator #2 Operator #3 Y/N Y/NSample # Attribute Try #1 Try #2 Try #1 Try #2 Try #1 Try #2 Agree Agree

1 pass pass pass pass pass fail fail N N2 pass pass pass pass pass fail fail N N3 fail fail fail fail pass fail fail N N4 fail fail fail fail fail fail fail Y Y5 fail fail fail pass fail fail fail N N6 pass pass pass pass pass pass pass Y Y7 pass fail fail fail fail fail fail Y N8 pass pass pass pass pass pass pass Y Y9 fail pass pass pass pass pass pass Y N

10 fail pass pass fail fail fail fail N N11 pass pass pass pass pass pass pass Y Y12 pass pass pass pass pass pass pass Y Y13 fail fail fail fail fail fail fail Y Y14 fail fail fail pass fail fail fail N N15 - - - - - - -

16 - - - - - - -

17 - - - - - - -

18 - - - - - - -

19 - - - - - - -

20 - - - - - - -

21 - - - - - - -

22 - - - - - - -

23 - - - - - - -

24 - - - - - - -

25 - - - - - - -

26 - - - - - - -

27 - - - - - - -

28 - - - - - - -

29 - - - - - - -

30 - - - - - - -

% APPRAISER SCORE(1) -> 100.00% 78.57% 100.00%

% SCORE VS. ATTRIBUTE(2) -> 78.57% 64.29% 71.43%

SCREEN % EFFECTIVE SCORE(3) -> 57.14%

SCREEN % EFFECTIVE SCORE vs. ATTRIBUTE(4) -> 42.86%


CalculationKnown Population Operator #1 Operator #2 Operator #3 Y/N Y/NSample # Attribute Try #1 Try #2 Try #1 Try #2 Try #1 Try #2 Agree Agree

1 1 1 1 1 1 2 2 FALSE FALSE2 1 1 1 1 1 2 2 FALSE FALSE3 2 2 2 2 1 2 2 FALSE FALSE4 2 2 2 2 2 2 2 TRUE TRUE5 2 2 2 1 2 2 2 FALSE FALSE6 1 1 1 1 1 1 1 TRUE TRUE7 1 2 2 2 2 2 2 TRUE FALSE8 1 1 1 1 1 1 1 TRUE TRUE9 2 1 1 1 1 1 1 TRUE FALSE10 2 1 1 2 2 2 2 FALSE FALSE11 1 2 1 1 1 1 1 FALSE FALSE12 1 2 1 1 1 1 1 FALSE FALSE13 2 2 2 2 2 2 2 TRUE TRUE14 2 2 2 1 2 2 2 FALSE FALSE15161718192021222324252627282930

% Appraiser Score 85.71% 78.57% 100.00% 42.9% 28.6%Agreements 12 11 14 6 4

Total Sample 14 14 14 14 14


Attribute Gage R&R Workshop

• Perform attribute Gage R&R study using M&M's.– Determine the defects by looking for consistent color in

the M&M's, clear markings (M's), and roundness.– Use 3 operators/inspectors

• Complete attribute GR&R analysis and report results (30 minutes).

• Improve inspection criteria, rerun attribute Gage R&R study/analysis and report results (30 minutes).


ANY QUESTIONS OR COMMENTS

Do You Understand?

• The language of Measurement ?

• The importance of Measurement?

• How to perform a Gage R&R Study and how to interpret results ?

• Use Minitab to analyze GR&R results?

Business

MSA - Measurement Systems Analysis