Measurement Measurement System AnalysisSystem Analysis

Kevin B. Craner

Boise State University

October 6, 2003

OverviewOverview

Purpose of measurement system analysis (MSA) Organizational uses of MSA Measurement system errors Accuracy and Precision of measurements MSA process flow Real world application Exercise

Measurement System Measurement System Analysis (MSA)Analysis (MSA)

aka Gauge R&R A tool used to evaluate the statistical properties of

process measurement systems. Purpose of MSA is to statistically verify that

current measurement systems provide:– Representative values of the characteristic being

measured– Unbiased results– Minimal variability

Organizational UsesOrganizational Uses

Mandatory requirement for QS 9000 certification.

Identify potential source of process variation.

Minimize defects.Increase product quality.

Measurement System ErrorsMeasurement System Errors

Accuracy: difference between the observed measurement and the actual measurement.

Precision: variation that occurs when measuring the same part with the same instrument.

Measurement System ErrorMeasurement System Error

Precise but not accurate

Accurate but not precise

Not accurate or precise

Accurate and precise

Accuracy of MeasurementAccuracy of Measurement

Broken down into three components:1. Stability: the consistency of measurements over time.2. Accuracy: a measure of the amount of bias in the system.3. Linearity: a measure of the bias values through the expected range of measurements.

Precision of MeasurementPrecision of Measurement

Precision, aka Measurement Variation, can be broken down into two components:1. Repeatability (Equipment variation): variation in measurements under exact conditions.

2. Reproducibility (Appraiser variation): variation in the average of measurements when different operators measure the same part.

MSA Process FlowMSA Process Flow

1. Preparation for study

2. Evaluate stability

3. Evaluate resolution

4. Determine accuracy

5. Calibration

6. Evaluate linearity

7. Determine repeatability and reproducibility

Preparation for StudyPreparation for Study

Objective: establish process parameter for the study.

Process:

1. Determine which measurement system will be studied.

2. Establish test procedure.

Preparation for StudyPreparation for Study-Cont--Cont-

3. Establish the number of sample parts, the number of repeated readings, and the number of operators that will be used.

4. Choose operators and sample parts.

Evaluate StabilityEvaluate Stability

Objective: evaluate measurement system to determine if the system is in statistical control.

Procedure: 1. Choose sample standards.2. Measure sample standards three to five

times.

Evaluate StabilityEvaluate Stability-Cont--Cont-

3. Plot data on a x-bar and R chart.

Analysis:

1. Determine if process is in control.

2. If process is unstable determine and correct the cause.

Evaluate ResolutionEvaluate Resolution

Objective: determine if the measurement system can identify and differentiate between small changes in the given characteristic.

Process:

1. Choose a sample standard.

Evaluate ResolutionEvaluate Resolution-Cont--Cont-

2. Measure the sample standard three to five times.

3. Repeat the process 10 to 25 times.

4. Plot data on a R chart.

Evaluate ResolutionEvaluate Resolution-Cont--Cont-

Analysis

1. The resolution is inadequate if:

- There are only one, two, or three possible values for the range, or

- There are only four possible values for the range when n >= 3.

Determine AccuracyDetermine Accuracy

Objective: determine the variation between the observed measurement and the actual measurement of a part.

Process:1. Choose sample standards.2. Measure sample standards 15 to 25 times using

the same measuring device, the same operator, and the same setup.

Determine AccuracyDetermine Accuracy-Cont--Cont-

3. Calculate x-bar

4. Calculate bias

- Bias = Average – Reference Value

5. Calculate the upper and lower 95% confidence limit (CL).

Determine AccuracyDetermine Accuracy-Cont--Cont-

Analysis

1. If reference value is within the 95% CL then the bias is insignificant.

2. If reference value is outside the 95% CL then the bias is significant and measurement system must be recalibrated.

CalibrationCalibration

Objective: to ensure the instrument is accurate, and measurement bias is minimized.

Process: calibrate instrument IAW manufacturers instructions.

Evaluate LinearityEvaluate Linearity

Objective: determine the difference between the obtained value and a reference value using the same instrument over the entire measurement space.

Process:1. Choose three to five sample standards that

cover the measurement space.

Evaluate LinearityEvaluate Linearity-Cont--Cont-

2. Measure sample standards 15 to 25 times.

3. Calculate the average of the readings.

4. Calculate bias.

5. Plot reference values on x-y graph.

6. Calculate slope of the linear regression line.

7. Calculate linearity and percent linearity.

8. Calculate R2.

Evaluate LinearityEvaluate Linearity-Cont--Cont-

Analysis

1. The closer the slope is to zero, the better the instrument.

2. R2 gives indication of how well the “best-fit” line accounts for variability in the x-y graph.

Determine Repeatability and Determine Repeatability and ReproducibilityReproducibility

Objective: determine variation in a set of measurement using a single instrument that can be credited to the instrument itself, and to the entire measurement system.

Process1. Generate random order for operators and

parts to complete the run.

Determine Repeatability and Determine Repeatability and Reproducibility -Cont-Reproducibility -Cont-

2. Repeat process for subsequent runs.

3. Have operators take measurements.

Analysis:

1. Plot data

2. Run ANOVA (analysis of variance) on data.

Determine Repeatability and Determine Repeatability and Reproducibility -Cont-Reproducibility -Cont-

3. Calculate total variance.

4. Calculate % Contribution and determine if acceptable.

5. Calculate % Contribution (R&R)

6. Calculate Process to Tolerance ratio (P/T) for repeatability.

7. Determine if P/T is acceptable.

Real World ApplicationReal World Application

Case study performed by Tirthankar Dasgupta and S.V.S.N Murthy, Indian Statistical Institute, 2001.- Gauge R&R study of automobile radiator manufacturer.- After studying four characteristics of radiator components the following results were obtained:

Table 1. Results of Table 1. Results of Preliminary StudyPreliminary Study

Characteristic Gauge

Least Count TV %EV %AV %R&R

Filler neck hole dia. Vernier calipers 0.01 0.1107 35.77 72.88 81.12

Tube crown thickness Dial vernier calipers 0.02 0.1866 49.19 38.06 62.1

Tube width Dial vernier calipers 0.02 0.0365 10.71 43.83 46.87

Inlet hole dia. Vernier calipers 0.01 0.1057 86.56 20.09 88.83

Source: Dasgupta & Murthy, Total Quality Management, Vol. 12, No. 6, 2001

Real World ApplicationReal World Application-Cont--Cont-

Any system having greater than 30% gauge R&R is considered inadequate. As seen in Table 1, all four characteristics’ %R&R is inadequate.

Investigation of the measurement system led to a subsequent reduction of %R&R in three of the four characteristics to between 12% and 23%.

Real World ApplicationReal World Application-Cont--Cont-

Further investigation of the fourth characteristic, inlet hole diameter, led the examiners to a manufacturing problem. The team discovered high ovality in the inlet hole, which was caused by the cutting tool. The tool was modified to reduce ovality.

Real World ApplicationReal World Application-Cont--Cont-

Benefits of the study

1. Reduced measurement variation.

2. Increased operator confidence regarding their aptitude for conducting gauge R&R studies.

3. Paved the way for further studies within the firm.

An Exercise – Calculating EV, An Exercise – Calculating EV, AV, R&R, and TVAV, R&R, and TV

Given: EV = 5.15(s0) , AV = 5.25(s1)

R&R = (EV2 + AV2)

TV = (EV2 + AV2 + PV2)Where: s0 = gauge standard deviation = 0.05

s1 = true appraiser standard deviation = 0.1

PV = part-to-part variation = 0.02

An Exercise – Calculating EV, An Exercise – Calculating EV, AV, R&R, and TVAV, R&R, and TV

Calculate R&R and TV

Is the calculated R&R acceptable?

SummarySummary

MSA studies are required for QS 9000 certification. However, MSA can prove beneficial to any firm that uses measurement systems whether they are seeking QS 9000 certification, or not.

MSA studies are a tool that aid in ensuring quality at all levels of a process.

SummarySummary-Cont--Cont-

According to David C. Crosby, “If you don’t know the capability of your measurement system, you don’t know if your measurements, or your products, are good or bad.”

ReferencesReferences

Crosby, David C. A Managers Guide to Gauge R&R, Rubber World 218 (1998): 16-17.

Goyal, Niraj. Selecting Appropriate Metrics. www.isixsigma.com/library/content.c020930a.asp. 4 Oct 2003.

Hemanth, S. Anomoly In Normality. www.isixsigma.com/library/content/c02820a.asp. 4 Oct 2003.

Niles, Kim. Characterizing the Measurement Process. www.isixsigma.com/library/content/c020527a.asp. 4 Oct 2003.

Measurement Systems Analysis Overview. http://mathstat.carleton.ca/~help/minitab/QCMEASYS.pdf. 4 OCT 2003.

Measurement System Analysis Resolution and Granularity. www.isixsigma.com/library/content/c000903a.asp. 4 Oct 2003.

ReferencesReferences

Measurement System Capability Manual. www.6sigma.us/MSASymbolsandNotation.html. 3 Oct 2003.

Dasgupta, Tirthankar & Murthy, S.V.S.N. Looking beyond audit-oriented evaluation of gauge repeatability and reproducibility: A case study. Total Quality Management. 12 (2001): 649-655.